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INTRODUCTION 


In  this  second  semiannual  report  we  describe  work  in  three  areas. 
Part  A  describes  our  work  on  packet  radio  networks.  This  section 
extends  our  work  to  include  the  effect  of  imperfectly  heard  acknowledg¬ 
ments.  It  describes  and  analyzes  a  new  protocol  that  reduces  this 
effect.  The  effect  of  the  increase  in  overhead  is  also  presented. 

In  Part  B  we  describe  our  work  on  new  switching  techniques. 
Here  we  prove  and  develop  an  optimum  packetization  strategy  for  a 
particular  class  of  users. 

In  Part  C  we  discuss  our  work  on  probabilistic  analysis  of  algor¬ 
ithms.  In  this  work  we  analyze  a  particular  algorithm,  and  discuss  its 
probabilistic  performance.  Simulation  results  are  also  presented. 

The  Appendix  contains  the  manuscript  of  a  paper  on  an  algorithm 
for  the  knapsack  problem.  It  is  being  submitted  for  publication. 
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A.  PACKET  RADIO  NETWORKS 

Our  activities  in  this  area  focused  on  developing  and  analyzing  a 
new  protocol  for  hop-by-hop  acknowledgments  in  multihop  packet  radio 
networks.  Our  objective  was  to  overcome  the  major  disadvantages  of 
the  passive  acknowledgment  scheme  previously  in  use.  In  particular, 
we  sought  to  improve  throughout  and  reduce  delay  relative  to  that 
scheme . 

We  developed  and  analyzed  a  new  acknowledgment  protocol  which 
we  refer  to  as  the  virtual  acknowledgment  protocol.  In  this  protocol,  if 
node  j  hears  a  packet  transmitted  to  it  by  node  i,  node  j  includes  the 
identifier  of  this  packet  in  the  next  m  packets  it  transmits  to  any  node 
(in  addition,  of  course,  to  later  transmitting  the  packet  itself).  We 
refer  to  this  as  m  virtual  transmissions  (and  one  actual  transmission)  of 
the  packet.  Node  i  then  has  m  opportunities,  rather  than  one,  to  hear 
that  node  j  received  its  packet.  Thus  the  probability  of  an  acknowl¬ 
edgment  being  heard  is  greatly  increased  over  what  it  was  with  passive 
acknowledgments.  This  increases  throughput  significantly,  roughly  50% 
for  the  topologies  we  have  studied. 

In  addition  to  this,  node  i  can  make  the  determination  as  to  whe¬ 
ther  or  not  to  transmit  as  soon  as  it  hears  any  packet  transmitted  by 
node  j.  Thus,  node  i  does  not  have  to  wait  for  its  packet  to  be  sched¬ 
uled  for  retransmission  and  retransmitted  by  node  j.  This  reduces 
delay  significantly. 

We  present  the  analysis  of  throughput  and  overhead  due  to  the 
increased  length  of  the  header  and  give  a  general  procedure  for  com¬ 
puting  these  quantities.  We  also  present  analyses  of  several  specific 
topologies  of  interest  and  compare  the  performance  of  this  protocol  with 
that  of  passive  acknowledgments  for  these  topologies. 
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A.l.  A  NEW  PROTOCOL  FOR  PASSIVE  ACKNOWLEDMENTS 


In  the  previous  report  we  discussed  the  effect  of  imperfect  ac¬ 
knowledgment  on  the  maximum  obtainable  throughput  for  various  top¬ 
ologies.  We  saw  that  the  maximum  obtainable  throughput  was  reduced 
more  than  30  percent  for  a  chain,  and  more  than  44  percent  for  the  star 
network.  The  protocol  that  we  used  has  the  following  disadvantages: 


1.  Each  node  has  only  one  chance  to  hear  acknowledgments 
from  its  neighbors. 

2.  Since  the  probability  of  a  node  hearing  an  acknowledg¬ 
ment  from  its  neighbors  is  significantly  less  than  one, 
some  of  the  successful  transmissions  are  duplicate  trans¬ 
missions  . 

3.  The  next  node  must  recognize  the  duplicate  transmission, 
otherwise  the  actual  throughput  will  go  to  zero  when  the 
number  of  hops  between  source  and  destination  nodes  is 
large . 

4.  Since  each  node  must  wait  until  transmission  of  the 
packet  by  its  neighbor  to  hear  acknowledgments  for  that 
packet,  the  timeout  period  is  large.  Thus  delay  will  be 
large  too. 


The  protocol  that  we  have  developed  to  overcome  these  problems  is 
that  a  node  recognizes  a  passive  acknowledgment  for  its  packet  by  the 
virtual  transmission  of  that  packet.  A  packet  is  virtually  transmitted  if 
its  identifier  is  transmitted  with  another  packet.  Each  node  when 
transmitting  a  packet  includes  not  only  the  packet's  identifier  but  also 
the  identifiers  of  previously  received  packets.  A  packet's  identifier  is 
contained  in  the  next  m  packets.  A  node  would  then  have  m  oppor¬ 
tunities  to  hear  if  its  transmission  was  received  (we  do  not  count  the 
actual  transmission).  If  a  packet's  identifier  is  transmitted  m  times 
then  the  acknowledgment  for  that  packet  is  almost  always  heard  for  a 
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sufficiently  large  value  of  m.  If  a  packet's  transmission  is  not  re¬ 
ceived,  then  after  a  timeout  period  that  packet  is  rescheduled  for 
transmission.  A  node  can  recognize  the  success  of  its  transmissions 
without  waiting  very  long  if  each  node  uses  a  FIFO  (a  first  in  first 
out)  discipline  for  the  virtual  transmissions.  Suppose  that  a  node 
hears  an  acknowledgment  for  packet  B ,  which  is  transmitted  after 
packet  A,  but  still  has  not  heard  a  virtual  transmission  of  A,  then  the 
node  recognizes  that  packet  A  was  not  received,  and  it  reschedules 
packet  A  for  retransmission. 


A. 2.  The  Effects  of  the  New  Protocol  on  Throughput 

Inserting  n  identifiers  into  a  packet  will  increase  a  packet  length 
by  a  factor  (see  Fig.  A.l) 


h  +  d  +  (n+1)  b 
h  +  d  +  b 


(A-l) 


The  value  of  n  is  related  to  m  as  will  be  shown  later.  Here  n  is  the 
number  of  different  identifiers  added  to  the  packet.  For  the  moment, 
we  will  assume  that  the  average  value  of  n  equals  m. 

The  value  of  m  is  chosen  such  that  the  probability  of  hearing  an 
acknowledgment,  q.j,  is  close  to  one.  Since  every  node  hears  acknowl¬ 
edgments  from  its  neighbors  with  probability  close  to  one  there  are 
almost  no  duplicate  transmissions.  Thus  nodes  do  not  have  to  recognize 
duplicate  transmissions  for  the  sake  of  throughput. 

If  each  node  virtually  transmits  a  packet  m  times  then  the  proba¬ 
bility  of  successfully  hearing  an  acknowledgment  can  be  recomputed  as 
follows : 
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FIGURE  A.l.  PACKET  FORMATS  FOR  THE  OLD  AND  NEW  PROTOCOLS 


q-j  is  the  probability  of  hearing  an  acknowledgment.  Then  the 
probability  of  not  hearing  an  acknowledgment  after  m  virtual  transmis¬ 
sions  is 


* 


■ 1  -  (1‘V 


m 


(A-2) 


^  £ 

For  large  values  of  m,  goes  to  1.  When  q.j  =  1  the  throughput 

(in  bits/sec)  will  increase  to  the  results  for  perfect  capture. 

* 

q.j  is  substituted  for  q.j  in  all  throughput  equations  for  chain  and 
store  networks  which  we  presented  previously.  The  maximum  obtainable 
throughput  for  different  topologies  are  given  in  Tables  A.l  through 
A. 3.  The  probability  of  hearing  acknowledgments  is  also  given  in 
Tables  A. 4  and  A. 5. 

We  see  from  Tables  A. 4  and  A. 5  that  the  probability  of  hearing 
acknowledgments  approaches  1  when  m  =  5  for  zero  connectivity  and 
m  =  4  for  full  connectivity.  The  throughput  almost  increases  to  the 
values  of  perfect  acknowledgment  for  m  =  5. 
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Table  A.l.  Effect  of  imperfect  acknowledgments  on  the  throughput 
as  a  function  of  M  for  the  chain  network  (S.j  =  S..  =  S). 


Length 
of  Chain 
(N) 

Throughput  of  a  chain 

number  of  virtual  transmissions,  m 

_ . _ 

Perfect 

Acknowledg¬ 

ment 

n 

2 

3 

4 

5 

6 

■ 

8 

4 

.106 

.119 

.124 

.127 

.127 

.1275 

.1276 

.128 

.128 

5 

.083 

.1 

.106 

.108 

.11 

.1105 

.1108 

.1109 

.111 

6 

.072 

.09 

.096 

.1 

.101 

.102 

.102 

.102 

.102 

7 

.066 

.085 

.091 

.095 

.096 

.097 

.097 

.097 

.097 

8 

.063 

.081 

.088 

.091 

.093 

.094 

.094 

.094 

.094 

9 

.061 

10 

.06 

00 

t" 

o 

.085 

00 

00 

o 

.09 

.09 

.0903 

.0905 

.091 

Table  A. 2.  Effect  of  imperfect  acknowledgments  on  the  throughput 

as  a  function  of  m  for  five  legs  in  a  star  network  with 
zero  connectivity  (S..  =  S-.  =  S). 


1 

I 

Length 
of  Legs 
(K) 

Throughput  of  a  star  with  five  legs 
number  of  virtual  transmissions,  m 

Perfect 

Acknowledg¬ 

ment 

1 

2 

3 

4 

5 

6 

1 

m 

.058 

.058 

2 

M 9 

.043 

.044 

.0445 

.045 

3 

.029 

.037 

El 

.042 

.043 

.0433 

.044 

>  4 

.029 

.037 

.04 

.0415 

.042 

.043 

.044 

Table  A. 3.  Effect  of  imperfect  acknowledgment  on  the  throughput 

as  a  function  of  m  for  six  legs  in  a  star  network  with 
full  connectivity  (S-  =  S..  =  S). 


Length 
of  Legs 

(K) 


Throughput  of  a  star  with  six  legs 
number  of  virtual  transmissions,  m 


Perfect 

Acknowledg¬ 

ment 


same  as  perfect  acknowledgment 
same  as  perfect  acknowledgment 
.023  .029  .033  .035  .036  .037 

.023  .029  .032  .034  .035  .036 


.083 

.042 

.04 

.039 


Table  A. 4.  Probability  of  hearing  acknowledgment  (q^)  as  a 

function  of  m  for  five- legs  of  length  5  in  a  star  net¬ 
work  with  zero  connectivity. 


number  of  virtual  transmissions,  m 


q. . 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10  j 

Qoi 

.530 

.700 

.810 

.86 

.91 

.94 

.95 

.97 

.99 

3l2 

.670 

.890 

.960 

.990 

.997 

.999 

1 

Q23 

.890 

.980 

.990 

.999 

1 

; 

.910 

.990 

.994 

.999 

1 

I  q4s 

1 

1 

1 

1 

Txo 

.990 

1 

T21 

.930 

.990 

.999 

1 

Q32 

.940 

.990 

.999 

1 

q43 

.999 

1 

I 

1 

qs4 

1 

j 

I 


i 
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We  compared  the  two  protocols  for  acknowledgments  and  tabulate 
the  results  in  Tables  A. 6  and  A. 7  for  chain  and  star  networks.  The 
throughput  of  the  center  node  increased  by  50%  for  a  ten  node  chain, 
by  48%  for  zero  connectivity,  and  by  more  than  50%  for  full  connec¬ 
tivity.  We  see  from  Table  A. 8  that  for  m  >  4  full  connectivity  retains 
its  superiority  with  respect  to  zero  connectivity. 

Table  A. 6.  Comparison  between  old  and  new  protocols  for  variable 
lengths  of  chains. 


Table  A. 7. 


Comparison  of  old  and  new  protocols  for  star  network. 
L=  number  of  legs;  K=  length  of  legs. 


Throughput  of  the  Center  Node. 


Connec- 

Imperfect  Acknowledgments 

O 

0  I 

tivity 

old  protocol 

new  protocol  (m=5) 

Increment  i 

Zero 

L  =  5 

.145 

.210 

j 

48  j 

K  >  4 

j 

Full 

L  =  6 

.138 

.210 

52 

K  >  4 

Full 

L  =  9 

.139 

.216 

56 

K  >  4 

Table  A. 8.  Comparison  of  connectivities  for  new  protocol. 


Connec¬ 

tivity 

Throughput  of  the  Center  Node 
number  of  virtual  transmission,  m 

1 

■ 

2 

3 

D 

5 

6 

7 

8 

Zero 

L  =  5 

K  >  4 

Full 

.145 

.185 

.20 

.206 

.210 

.215 

.2155 

.216 

L  =  6 
k  >  4 

Full 

.128 

.174 

.192 

.204 

.210 

.216 

.222 

.224 

L  =  9 

K  >  4 

.139 

.176 

.198 

.207 

.216 

.225 

■ 

.230 

.234 

As  we  mentioned  before  the  length  of  a  packet  is  increased  by  a 
factor 

f  _  h  +  d  +  (m+1)  •  b 
h  7  d  +  b 


-8- 


because  of  the  virtual  transmissions.  We  inserted  m  •  b  extra  bits  into 


a  packet.  Thus,  the  throughput  is  given  by 


Si;  =  G. 


P(Ni#  N.) 


* 


h  +  d  +  b 
h  +  d  +  (m+1)  •  b 


If  we  choose  b  =  12,  h  =  84,  m  =  5,  and  d  =  960  bits  then 


h  +  d  +  b 
h  +  d  +  (m+1)  •  b 


.946 


or 


m  ♦  b  _ 
h  +  d  +  b  _ 


.057 


(A-3) 


This  means  that  the  throughput  decreased  6%  when  compared  with  per¬ 
fect  acknowledgments.  But  we  know  from  Tables  A. 6  and  A. 7  that  the 
throughput  increased  much  more  than  6%  because  of  the  new  protocol. 
The  results  in  Tables  A.l,  A. 2  and  A. 3  were -obtained  by  using 


V 


Gu 


P(N,,  Nj) 


(A-4) 


In  order  to  incorporate  Eq.  (A-3),  the  results  in  Tables  A.l,  A. 2 
and  A. 3  are  divided  by  Eq.  (A-l). 


A. 3.  Optimum  m  for  the  New  Protocol 

In  the  previous  section  we  have  studied  the  effect  of  the  new 
protocol  on  throughput.  We  found  that  for  m  >  4  the  star  network  with 
full  connectivity  retains  its  superiority  with  respect  to  the  other  con¬ 
nectivities.  In  this  section  we  will  more  closely  study  the  L-leg  star 
network  with  full  connectivity. 

* 

We  know  that  increasing  m  will  increase  q..  and  hence  throughput. 
There  is,  however,  a  tradeoff  between  the  throughput  and  overhead 
(due  to  the  increase  in  the  packet's  length)  due  to  virtual  transmissions. 
For  several  cases  we  will  find  the  optimum  m.  First  we  assume  that  at 
all  nodes,  if  a  packet's  identifier  is  transmitted  virtually  m  times,  then 


we  also  insert  m  identifiers  of  received  packets  into  the  header  of  a 
packet  which  is  ready  for  transmission.  The  next  m  transmissions, 
after  receipt  of  the  packet  in  question  will  include  the  identifier  of  that 
packet. 

For  this  case  the  throughput  is  given  by 


S..  =  G-. 
i)  i] 


P(Nj,  N.) 


h  +  d  +  b 
h  +  d  +  (m+l )  ~ B 


(A-5) 


where  j  is  the  one  of  i's  neighbors. 

The  solution  of  Eq.  (5)  for  the  9-leg-star  network  is  given  in 
Table  A-9  when  k  >  4.  The  optimum  m  is  10  for  this  case. 


Table  A. 9  Total  throughput  of  a  hotspot  for  a  star  network  (L  =  9, 
K  =  4)  when  increment  in  header  of  node  i,  I.  =  m  •  b, 
i  =  0,1,2 .  K.  1 


m 

sT 

1 

.1323 

2 

.1682 

3 

.1886 

4 

.1961 

5 

.2025 

6 

.2091 

7 

.2130 

8 

.2145 

10 

.2146 

11 

.2142 

12 

.2138 

15 

.2110 

20 

.2045 

In  Eq.  (A-l)  we  assume  that  the  increment  in  a  packet's  length  is 
m  •  b  bits.  But  we  know  that  the  hotspot  (node  0)  and  nodes  at  the 
end  of  the  legs  do  not  have  to  send  passive  acknowledgments  because 
the  transmissions  into  the  hotspot  and  nodes  at  the  end  of  the  legs  will 


never  have  collisions.  Thus  there  is  no  increment  in  the  packet's 
length  for  these  nodes.  Equation  (A-5)  can  be  modified  to  obtain  exact 
results  as  follows: 

Again  we  assume  that  node  0  sends  S  units  of  traffic  in  each 
direction.  Then  the  nodes  that  have  m  •  b  bits  increment  in  their 
packet's  length  must  send  I  •  S  units  of  traffic  to  their  neighbors 
where 


T  _  h  +d+  (m+l)b 
h  +  d  +  b 


For  left  to  right  transmissions  the  throughput  equations  are: 


Soi  =  S  =  G01  •  P(N0,  NO 

si,w  =  1  •  s  =  Gun  '  p(Ni-  nm> 

For  right  to  left  transmissions: 

Sfc  jc_2  =  S  = 


n,i+l 


for  i  =  1,2, ... ,  K-l 
(A-6) 


SM.i  *  1  •  S  =  GW,i  •  P<Ni-  »M>  '  Vl.i  f°r  1  *  "-I .  ^ 


We  expect  that  the  results  from  Eq.  (A-6)  are  greater  than  the 

results  from  Eq.  (A-5)  because  Eq.  (A-5)  is  solved  assuming  that  the 

packet's  length  is  incremented  by  m  •  b  bits  at  all  nodes. 

The  results  for  Eq.  (A-6)  are  given  in  Table  A-10.  We  see  from 

Table  A-10  that  the  optimum  m  is  11  and  the  total  throughput  of  the 

hotspot,  S^,  is  .2222.  When  m  >  11  the  throughput  decreases  because 

of  overhead  due  to  virtual  transmissions. 

* 

We  know  from  Table  A-5  that  q..  =  1  when  m  =  4  for  all  i  and  j 
* 

except  q12.  This  means  that  nodes  1,3,4,...,  k-l  need  transmit  a 
packet's  identifier  only  four  times.  Let  us  assume  that  node  2  trans¬ 
mits  m'  times  and  all  the  other  nodes  transmit  m  times.  Then  the 
packet's  length  is  incremented  by 


Table  A.  10. 


Total  throughput  of  the  hot  spot  in  a  star  network 
(L  =  9;  K  =  4)  when  increment  in  header  I0  =  I,  =  0 ; 
I.  =  m  •  b,  i  =  1,2,3 .  k-1 


1.  =  m  •  b  bits 

12  =  m  •  b  bits 


for  i  =  1,3,4, ... ,  k-1 


s  sk,k-l  Gk,k-1 


<Nk'  Nk-1> 


where 


t  _  h  +  d  +  (m+l)b 

i  -  - c — ,  J  .  1- — 


T -  _  h  +  d  +  (m'+l)*b 


(A-7) 


1(5  =  !k  ~  *2k  =  ’ '  *  ~  !LK  =  0 

With  these  assumptions  Eqs.  (A-6)  become 

for  left  to  right  transmissions 

S  =  S01  =  G01  •  P(N0,  Nx) 

1  ‘  s  =  Gi  i+1  •  P(Ni#  Ni+1)  •  q*  i+1 ,  i  =  1,3,4 .  k-1 

I'  •  S  =  G23  •  P(N2 ,  N3)  ♦  q23 

for  right  to  left  transmissions  we  have: 

1  ’  s  =  Gi+U  •  P(N.,  N.+1)  •  q*+1>i,  i  =  0,1, 2, 3 .  k- 

I'  •  S  =  G21  •  P(Nx,  N2)  •  q21 


>(A-8) 


(A-9) 


m\ 


c  a_1n^ 


For  h  =  84,  b  =  12,  d  =  960,  and  m  =  4  the  results  for  Eq.  (A-8) 
are  tabulated  in  Table  A.  11.  The  optimum  m'  is  19,  and  the  total 


throughput  of  the  hotspot,  S,p,  is  .2390.  With  m  =  4  and  m'  =  19  all 
* 

q.j  =  1,  therefore,  the  nodes  will  not  send  any  significant  number  of 
duplicate  transmissions  to  the  next  hop.  Since  for  node  2,  m'  =  19, 
the  increment  in  a  packet's  length  there  is  22%  (228  bits).  For  nodes  1 
and  3,  the  increment  in  the  packet's  length  is  4.5%  (48  bits). 


Table  A.  11.  Total  throughput  of  the  hotspot  in  a  star  network 
(m  =  4,  L  =  9,  K  =  4)  when  increment  in  header  length 
1°  =  lk  =  0;  12  =  nT  •  b;  1.  =  m  •  b,  i  =  1,3,4,...,  k-1 


m' 

ST 

10 

.2306 

11 

.2327 

12 

.2340 

IS 

.2370 

18 

.2383 

19 

.2390 

20 

.2390 

21 

.2390 

30 

.2383 

We  can  further  increase  the  throughput  by  reducing  overhead  as 
follows : 


A  node,  i,  receives  SR(.^,  units  of  the  successful  transmission 
(throughput)  from  its  neighbors  where 


SR(i)  = 


(A-ll) 


For  each  unit  of  successful  transmission  node  i  sends  m^  passive  ac¬ 
knowledgments  so  tha*  he  total  number  of  acknowledgments  at  node  i  is 
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mi  *  ^R(')  Per  un^  ^me-  But  n°de  i  transmits  with  a  rate  of 

G.  •  P(N.)  and  we  know  that  G,.  •  P(Nf)  >  SD/;-. .  Therefore  each 
11  1  1  —  rv^l ) 

transmission  must  carry  r.  passive  acknowledgments  where 


,=  i 

1  Gi 


SR(i) 

"Kntt 


(A-12) 


The  overhead  in  a  packet's  length  at  node  i  is  given  by 
‘i  =  ri  • b  •  for  1  *  °'i'2 . k- 


If  we  assume  that  the  hotspot  (node  0)  and  nodes  at  the  end  of 
each  path  (leg)  want  to  send  S  packets/unit  time  then  the  throughput 
is  given  by 


h  +  d  +  r.  •  b  * 

Sij  =  . -TTT-a-T-ir  •  3  =  Gij  •  Wi'  V  *  *!,  (A-!3) 

for  i  =  0,1,...,  k,  and  where  j  is  a  neighbor  of  node  i. 

For  instance  for  the  transmissions  from  left  to  right  we  have: 
when  i  =  0 

S01  =  S  =  G01  •  P(N0,  Nx)  •  q01 
i  =  1 

Soi  +  S21 

h  +  d  +  m  •  (G-Vp(Nl))  •  b 

SX2  = -  •  S  =  G12  •  P(Nlf  N2)  •  q12 

h  +  d  +  b 


h  +  d  +  m'  ( 


'12 


+  S 


32 


>23 


jUk y 


h  +  d  +  b 


S  =  G23  •  P(N2,  N3)  •  q23 


i  =  k-1 
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FIGURE  2.  FLOWCHART  TO  SOLVE  EQ.  (A-13) 


The  results  for  Eq.  (A-13)  are  given  in  Table  A.  12.  We  see  that 
the  total  throughput  of  the  hotspot,  S^,  is  .2454  and  the  optimum  m'  is 
27.  Even  though  node  2  transmits  a  packet's  identifier  27  times  the 

increment  in  the  packet's  length  is  only  78  bits  which  is  a  7%  incre¬ 

ment.  This  means  that  at  node  2  each  transmission  carries  6.5  identi¬ 
fiers  (r2  =  6.5).  We  also  found  the  rx  =  3.79  and  r3  =  3.82  which  are 

4%  increments.  In  the  previous  report  we  saw  that  there  was  a  46%  re¬ 

duction  in  throughput  due  to  imperfect  acknowledgments  when  we  used 
the  old  protocol  (no  virtual  transmissions).  Here  we  see  that  with  the 
new  protocol  there  is  only  a  2.6%  reduction  in  throughput  with  respect 
to  the  result  for  perfect  acknowledgments  (.252).  Up  to  this  point  we 
assumed  that  the  original  length  of  a  packet  is  1056  bits  (header  =  96, 
data  =  960). 


Table  A.  12  Total  throughput  of  the  hotspot  in  a  star  network 
(m  =  4,  L  =  9,  k  =  4)  when  In  =  1,  =  0;  1.  =  r.  •  b, 


The  effect  of  the  overhead  depends  also  on  the  length  of  the  data 
portion  of  a  packet.  If  the  data  portion  of  a  packet,  d,  is  small  the 
overhead  will  be  large  and  the  throughput  will  be  reduced.  We  have 
solved  Eq.  (A-13)  for  different  values  of  d.  The  results  are  given  in 
Table  A.  13.  In  Table  A.  13  we  also  tabulated  the  reduction  from  the 
result  for  perfect  acknowledgments.  We  see  from  Table  A.  13  that  when 
d  increases  the  throughput  also  increases  and  the  overhead  gets  smaller. 
The  overhead  and  r.  are  given  in  Table  A.  14.  When  d  =  50  bits  the 
overhead  at  node  two  is  36%  but  when  d  =  2000  the  overhead  is  only  4%. 
The  reduction  in  throughput  from  the  result  of  perfect  acknowledgment 
is  18%  when  d  =  50  but  it  is  only  .9%  when  d  =  2000. 


Table  A.  13.  Total  throughput  of  the  hotspot  in  a  star  network 
(m  =  4,  L  =  9,  K  =  4)  for  different  values  of  d. 


d 

m ' 

ST 

%  Reduction  from  perfect 

acknowledgments 

50 

16 

.2074 

18 

100 

20 

.2173 

14 

200 

15 

.2243 

11 

400 

19 

.2355 

7 

600 

22 

.2405 

5 

800 

27 

.2440 

3 

960 

27 

.2454 

2.6 

1200 

30 

.2468 

2 

1600 

35 

.2489 

1 

2000 

1 _ 

35 

.2496 

.9 

Table  A.  14.  The  percentage  of  overhead  as  a  function  of  d. 


d 

r 
r  1 

over¬ 

head 

% 

r  2 

over¬ 

head 

% 

To 

3 

■ 

over¬ 

head 

o 

0 

— 

i 

50 

3.49 

29 

4.35 

36 

“1 

3.5 

29 

m m 

3.55 

22 

5 

30 

3.6 

22 

Km 

3.56 

14 

4.36 

18 

3.65 

15 

■ 

5.16 

13 

7 

17 

5.25 

13 

■ 

3.74 

6 

5.57 

10 

3.8 

7 

Kill 

3.79 

5 

6.5 

9 

3.82 

5 

960 

3.79 

4 

6.5 

7 

3.82 

4 

■B&i-M 

3.83 

3 . 5 

7.8 

7 

3.84 

3.5 

3.83 

3 

9 

6 

3.84 

3 

3.84 

2 

8 

4 

3.85 

2 

where : 


r.  •  b 

overhead  =  +  d~  +~'b  *  100  '  ^  =  84,  b  =  12 


B.  NEW  SWITCHING  TECHNIQUES 

Our  research  in  this  area  focused  on  developing  an  optimal  packet¬ 
ization  scheme  for  situations  where  conventional  packetization  techniques 
are  unsuitable.  In  particular  we  seek  to  maximize  the  number  of  users 
which  could  share  a  channel  of  given  capacity  subject  to  a  constraint  on 
the  average  per  character  delay,  where  delay  includes  packetization 
delay  as  well  as  queuing  delay.  We  are  particularly  interested  in  cases 
where  the  packetization  delay  is  significant  and  the  delay  constraint  is 
significant.  An  example  of  such  a  situation  is  the  case  where  many 
interactive  users  access  a  packet  switched  network  with  low  speed 
access  links  and  require  an  echo  from  a  remote  host  of  the  characters 
they  enter. 


We  have  obtained  on  optimal  scheme  and,  in  this  section  give  a 
proof  of  its  optimality.  The  scheme  is  surprisingly  simple.  One  ob¬ 
tains  via  an  elementary  search  procedure  the  optimal  value  of  y,  the 
average  packet  length;  i.e.,  the  value  of  y  which  maximizes  M,  the 
number  of  users  in  the  system.  In  general,  y  is  not  an  integer.  We 
then  packetize  with  probability  1  -  q  if  n  characters  have  arrived  and 
with  probability  q  if  n  +  1  characters  have  arrived,  where  y  =  n  +  q. 
The  procedure  is  easy  to  implement  in  practice  and  is  significantly 
better  than  either  packetizing  on  a  fixed  number  of  characters  or  after 
a  fixed  amount  of  time. 

B.l  An  Optimal  Packetization  Strategy 

We  consider  the  problem  of  determining  the  optimal  strategy  for 
packetization  in  a  multi-user  packet  switched  network.  The  objective  is 
to  maximize  the  number  of  users,  M,  in  the  system  subject  to  a  restric¬ 
tion  on  the  average  per  character  delay.  This  problem  and  the  input 
model  we  use  are  motivated  by  interactive  data  entry  applications  where 
remote  echoing  of  each  input  character  is  required.  It  is  common 
practice  in  such  situations  to  packetize  on  every  character  or  on  the 
basis  of  a  very  short  timer.  The  result  of  this  is  that  very  small 
packets  are  sent.  This  is  inefficient  as  the  packet  header  than  repre¬ 
sents  a  significant  overhead,  often  in  excess  of  1000%.  This  has  often 
led  such  users  to  abandon  remote  echoing  (or,  alternatively,  to  abandon 
packet  switching).  We  here  offer  a  third  choice. 

We  assume  that  each  user  produces  a  character  with  probability  p 
every  t  seconds.  Characters  input  by  user  i  are  stored  in  buffer,  b., 
until  it  is  determined  on  the  basis  of  a  packetization  strategy  that  a 


packet  should  be  formed.  At  this  point,  a  header  of  length  h  is  added 
to  the  data  characters  and  the  resulting  packet  is  placed  on  a  queue 
for  transmission  on  a  channel  with  capacity  c  characters  per  second. 

The  average  per  character  delay,  D  is  given  by 

13  =  Cp  +  Cs  +  Cw  CB-1) 

where  D  ,  Ds,  and  Dw  are  the  average  per  character  packetization, 
service,  and  queueing  delay,  respectively.  All  times  are  measured  in 
slots  of  length  t. 


B.2  Packetization  Delay 

We  assume  that  characters  arrive  independently.  Each  character 
in  a  packet  waits  for  all  the  remaining  characters  to  arrive.  If  the 
packetization  scheme  results  in  a  packet  length  distribution  of 
f.  =  Frob  {Packet  length  is  i  characters} 
then  Dp,  the  average  per  character  packetization  delay,  is  given  by 

Dp  -  I  f,  V? 


where  dj  is  the  total  average  packetization  delay  suffered  by  all  charac¬ 
ters  in  a  packet  of  length  i  and  y  is  the  average  packet  length,  y  =  If.i 
Since  characters  arrive  independently  at  a  rate  of  p  characters  per 
slot,  the  average  interarrival  time  between  characters  is  1/p  slots 
and  d.  obeys  the  recurrence  relation 


a.  =  a.  ,  +  i=i 

i  i-i  p 

This  together  with  the  initial  condition  d.  =  0,  yields 


i(i-l) 

2p 
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Thus,  for  example,  if  all  packets  were  n  characters  long 


D  =n  1-1 
P  2p 


(B-2) 


and  if  packets  were  n  characters  long  with  probability  1  -  q  and  n  +  1 
characters  long  with  probability  q,  so  that  the  average  packet  length 

y  is  n  +  q,  then 


2p(n+q) 

Thus  we  see  in  this  case  that  Dp  is  linear  with  y  for  integer  values  of 
y  and  is  the  continuous  piece  wise-differentiable  function  given  in 
Figure  B.l.  Note  that  the  function  is  indeed  continuous  and  matches 
the  function  for  integer  values  of  y  since  q(l-q)  =  0  for  q  =  0  or 
1.  Note  that  the  maximum  difference  between  D  and  ^  occurs  at 

1  P  .  2p 

q=*5  and  equals  — _and  that  this  value  decreases  as  y  increases. 


Fig  B.l  Average  Packetization  Delay  for  2-point 
Packet  Length  Distribution,  y  =  n+q 


Cj/-l 


n=l,d=0  n=l,q=l  or  n=2,q=0 


I 

B.4  Service  Time 

i  The  average  service  time  per  character  Ds  given  by 

D  =  [(I  f.  i2/y)  +  h]/c 

s  j  i 

I  where  c  is  the  high  speed  line  speed  in  characters/slot.  Thus  for 

constant  length  packets  of  length  n 

D  =  (n  +  h)/C 

I 

1  and  for  the  2-point  distribution  given  above  with  parameters  n  and  q 

n  -  n  +  q  +  h  +q(A-q)  _ 

s  c  c(n  +  q) 

1  Thus  we  see  that  D  behaves  exactly  the  same  way  D  does.  It  is 

s  P 

continuous,  piecewise-differentiable,  and  follows  closely,  the  error 

getting  smaller  as  y  increases. 

! 

i 

B.5  Waiting  Time 

The  delay  Dw  due  to  waiting  time  per  character  is  a  function  of 
I  the  distribution  of  the  interarrival  time  between  packets.  We  have 

carried  out  an  exact  analysis  of  the  waiting  time  for  fixed  length 
packets  and  obtained  an  expression  relating  it  to  the  packet  length 
*  (see  the  previous  semiannual  report).  Unfortunately,  we  have  been 

unable  to  obtain  a  closed  from  for  the  relationship  except  for  y  =  1  and 
y  =  2.  The  results  in  these  cases  support  the  approximation  of  Dw  by 
8  the  M!G!1  queuing  delay  formula.  We  will  thus  make  the  assumption 

that  Dw  can  be  approximated  by 

1  M  AX* 

2c2(l-p) 
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where  M  is  the  total  number  of  active  users 


A  is  the  average  arrival  rate  per  user  in  packets/siot;  thus  A 

=  i/y 

X2  is  the  second  moment  of  the  service  time 
p  is  the  average  channel  utilization 


X2  can  be  written  as 


X2  =  2  fi(i  +  h)2/y 


For  constant  length  packets  this  is  simply  (n+h)2/c2  and  for  the 
2-point  distribution  above  is 


^  =  (n_+_q  +h)2  +  q(l-q) 

r*  2  Z 


The  channel  utilization,  p,  is  given  by 


We  thus  have  an  expression  for  the  total  delay  per  character,  D, 


D  =  i  f  [  Ki-l)  +  i2/y  +  h  +  MX(i-t-h)2  ^ 
*  2py  c  2c2(l-p)y 


(B-3) 


B.6  Derivation  of  the  ODtimum  Packetization  Strate 


We  now  consider  the  problem  of  maximizing  M,  the  number  of 
users,  subject  to  a  constraint,  D*,  on  the  average  per  character  total 
delay.  We  thus  seek  the  optimal  value  of  the  packet  length,  y,  and  the 
optimal  packetization  scheme  which  will  yield  this  optimal  length. 

Given  M  and  y,  we  see  from  equation  (B-3)  that  the  packetization 
scheme  which  minimizes  D  is  one  which  minimizes  the  second  moment  of 
the  packet  length,  Zi2 f . .  We  are  constrained,  of  course,  to  packet 
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length  distributions  where  the  packet  length  only  takes  integer  values. 
It  is  easy  to  see  that  the  distribution  which  minimizes  the  second  mo¬ 
ment  subject  to  these  constraints  is  the  2-point  distribution  which  we 
have  been  discussing;  i.e., 

1-q  i  =  n 

f.  =  q  i  =  n+1 

0  otherwise 

where  y  =  n+q.  To  see  this  we  use  the  technique  of  Lagrange  multi¬ 
pliers  to  find  the  minimum  of 

Ii2f.  +  a  lifj  +  If.  where 

a  and  0  are  the  Lagrange  multipliers.  The  last  two  terms  are  used  to 
satisfy  the  constraints  lif.  =  y  and  If.  =  1.  In  addition  all  f .  >  0 . 
Differentiation  with  respect  to  the  f;  we  get 

(  =  0  for  all  i  such  that  f .  >  0 
i2  +  ai  +  0  1 

(  >  0  for  all  i  such  that  f .  =  0 

Thus  there  are  at  most  two  non-zero  values  of  f.  that  can  satisfy  the 
equality  condition  above.  To  satisfy  the  inequality  condition  these  two 
values  of  i  must  differ  by  one. 

Thus  we  see  that  for  a  given  value  of  y  and  M,  the  total  delay  is 
minimized  by  any  packetization  scheme  which  gives  rise  to  a  2-point 
packet  length  distribution. 

We  approach  this  problem  in  a  different  way  by  considering  packet¬ 
ization  delay  and  service  time  together.  Any  packetization  scheme  can 
be  thought  of  as  a  decision  rule  which,  given  the  state  of  the  buffers, 
decides  whether  to  packetize  or  not.  Figure  B.2  shows  the  state  space 
for  this  decision  process.  The  root  of  the  binary  tree  shown  in  Figure 
B.2  represents  the  state  where  the  buffer  is  empty  and  no  time  has 
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Fig.  B.2  The  State  Space  for  the  Packetization  Decision  Process 

elapsed.  Note  that  the  packetization  rule  considers  each  user  sepa¬ 
rately  and  hence  the  buffer  is  that  for  a  single  user.  Each  link  repre¬ 
sents  a  transition  in  a  single  slot  time.  A  transition  toward  the  left  (to 
a  shaded  node)  occurs  with  probability  P  and  corresponds  to  the  arrival 
of  a  character.  A  transition  to  the  right  corresponds  to  a  slot  in  which 
no  character  arrives.  Tracing  a  path  from  the  root  to  any  node,  j,  one 
can  determine  the  probability,  Tj,  of  reaching  that  state  by  multiplying 
all  of  the  link  probabilities.  One  can  also  find  the  number  of  charac¬ 
ters,  nj,  in  the  buffer  by  counting  the  number  of  shaded  nodes  in  the 
path  from  the  root  to  node  j.  Finally,  one  can  determine  dj ,  the  total 
delay  due  to  packetization  and  service  time  suffered  by  all  characters  in 
the  buffer,  if  a  packet  is  formed  at  node  j.  This  is  done  by  adding 
together  the  delays  of  all  the  characters  in  the  packet. 

Let  Xj  be  a  decision  variable  taking  the  value  1  if  we  packetize  at 
node  j  and  zero  if  we  do  not.  Any  packetization  scheme  can  be  defined 
solely  in  terms  of  the  x. .  Furthermore, 
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and 


y  =  I  x.  r.  n 


J 


)  )  J 


s‘7  Vidi 


where  D  l  is  the  average  packetization  and  service  delay  for  all  charac¬ 
ters.  We  seek  to  maximize  y  subject  to  the  constraint  Di/y  <  D*.  The 
constraint  satisfies  the  time  delay.  Making  y  larger  increases  the 
channel  efficiency.  There  is  also  an  additional  constraint  that  the 
packetization  scheme  be  consistent.  In  particular,  x.  =  1  implies  that 
x^  =  0  for  all  successors  k  of  node  j  since  once  we  packetize  at  node  j 
we  will  never  reach  node  k. 

It  is  somewhat  easier  to  see  what  the  optimal  policy  is  if  we  con¬ 
sider  the  decision  process  in  a  slightly  different  way.  We  define  a  new 

set  of  decision  variables,  w. ,  associated  with  the  nodes  j.  The  Wj  are 
related  to  the  Xj  above  by  the  rule  that  if  Xj  =  1  then  Wj  =  1  for  all 
predecessors  of  j  and  Wj  =  0  otherwise.  For  all  consistent  values  of  Xj, 
the  w^.  are  well  defined.  Setting  w  =  1  can  be  thought  of  as  deciding 
not  to  packetize  at  node  j. 

We  now  define  gj  as  the  increment  in  packet  length  due  to  the 

decision  not  to  packetize  at  node  j.  Similarly  we  define  hr  as  the 

increment  in  the  average  total  delay,  Dt. 

The  expressions  for  y  and  D  x  can  then  be  rewritten  in  terms  of 
the  new  decision  variables,  Wj,  as  follows: 

Y  "  *  w.  g. 

j  J  J 

Dj  =  I  w.  h. 

j  J  ] 
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A 


The  incremental  effect  of  not  packetizing  at  node  j  is  shown  in 
Figure  B.3.  As  can  be  seen,  two  nodes,  k  and  1,  will  be  counted  in 


the  sums  for  y  and  Dx  instead  of  node  j. 


g.  =  (l-q)r. 


2n:  +  h  +  1 


hi =  <nj +  -Lr 


")ri 


In  order  to  maximize  y  for  a  given  D  we  would  prefer  to  choose 
nodes  j  with  maximal  g^/hj,  i.e.  nodes  which  maximize  the  incremental 


rk=a-q)rj  r,  =  qr 


W1 

dk=drni 


n,  =  n, 
d,  =d,  ♦  n, 


+2n|  h  1 


Fig.  B.3  Incremental  Effect  of  Not  Packetizing  at  Node  j 


gain  in  y  per  unit  D  x .  Note  that  in  reality  it  is  Dx/y  not  Dx  which  is 
the  constraint  but  since  we  seek  k  to  mimize  y  we  are  simultaneously 
loosening  the  constraint  as  much  as  possible. 

In  order  to  maximize  we  choose  nodes  j  with  the  smallest 

possible  n.  and  set  w.  =  1  for  those  nodes.  We  continue  to  pick  nodes 
until  we  find  that  setting  the  next  Wj  =  1  would  cause  Dx/y  to  exceed 
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D*.  Until  this  point,  we  have  assumed  for  the  sake  of  simplicity  that 
Wj  would  only  take  the  values  0  or  1.  In  fact  we  are  free  to  assign 
any  value  in  the  range  0  to  1  to  each  w. .  At  the  point  where  setting 
w.  =  1  would  violate  the  constraint  on  D1(  there  exists  a  value,  a,  of 
w.  which  would  cause  the  constraint  to  be  satisfied  exactly,  since  D*y 
is  a  monotore  function  w. .  Thus,  we  set  w.  =  a  for  this  last  j  and 
satisfy  the  constraint  with  equality. 

It  is  clear  that  this  procedure  will  indeed  find  an  optimal  set  of 
values  for  the  Wj  in  the  sense  that  y  is  maximized.  To  see  this  we 
need  only  note  that  in  order  to  increase  the  value  of  any  Wj  above  the 
value  set  by  this  procedure,  we  would  have  to  correspondingly  decrease 
the  value  of  some  other  w^  with  a  resulting  decrease  (or  at  best  no 
change)  in  y.  This  is  because  we  are  selecting  the  nodes  j  order  of 
gj/hj,  largest  first. 

Note  that  at  any  stage  there  is  an  infinite  number  of  nodes  j  with 
the  same  value  of  n^  to  choose  from.  Any  selection  will  yield  exactly 
the  same  value  of  y.  Indeed,  one  need  only  ask  if  Wj  can  be  set  to  1 
for  all  nodes  with  a  particular  nj.  If  so,  we  do  this  before  asking  the 
question  for  the  next  larger  value  of  n. .  Thus  we  avoid  any  conceptual 
difficulty  with  the  finiteness  of  the  decision  process. 

We  are  left  then  only  with  the  question  the  consistency  of  the 
packetization  policy.  This  too,  however,  is  no  problem.  Note  that  if 
we  set  Wj  =  1  if  a.  <  n*  and  also  for  a  collection  of  nodes  with  n.  =  n* 
and  with  aggregate  probability  1-q  then  we  have  a  consistent  packeti¬ 
zation  policy  which  adheres  to  the  optimization  criteria.  This  policy  can 

be  simply  stated,  and  equivalently  implemented,  as  waiting  until  the 
th 

n*  —  character  arrives,  flipping  a  coin  which  comes  up  heads  with 
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probability  q,  packetizing  at  that  point  if  the  coin  comes  up  tails,  and 
packetizing  after  the  next  character  arrives  otherwise. 

We  have  proven  that  given  y,  the  above  packetization  scheme  is 

optimal.  We  now  seek  the  value  of  y  which  maximizes  M,  the  number  of 

users.  For  y  between  any  two  integers,  n  and  n+1,  D  is  unimodel  and 

any  simple  search  procedure,  e.g.  binary  search,  suffices  to  minimize  D 

or  alternatively  maximize  M.  The  small  "bumps"  introduce  by  the 

q(l-q)  terms  in  D  and  D  necessitate  that,  strictly,  we  search  each 

P  S 

unit  interval  separately.  In  practice  only  a  small  number  of  intervals 
will  need  to  be  examined  for  most  values  of  c  and  D  because  the  q(l-q) 
term  will  usually  be  insignificant  in  comparison  with  the  variations  due 
to  a  unit  change  in  y.  Thus,  the  overall  search  is  computationally 
reasonable . 

Thus,  we  can  obtain  an  optimal  packetization  strategy  for  any 
given  D.  Computational  experience  indicates  that  for  small  D  and  large 
c  this  optimal  strategy  outperforms  ordinary  packetization  schemes  by 
as  much  as  20%. 

C.  PROBABILISTIC  ANALYSIS  OF  ALGORITHMS 

We  have  obtained  results  in  this  area  indicating  that  even  rela¬ 
tively  simple  algorithms  will,  with  high  probability,  obtain  near  optimal 
solutions  to  certain  difficult  optimization  problems.  This  is  encouraging 
in  that  it  implies  that  problems,  previously  considered  intractable,  may 
in  fact  be  solvable,  at  least  with  high  expectation  of  success.  The 
results  are  also  useful  in  guiding  heuristics  towards  potentially  fruitful 
areas  within  the  solution  space  and  in  avoiding  unnecessary  effort  in 
areas  where  little  further  progress  can  be  expected. 


Specifically,  we  have  analyzed  the  intrinsic  probabilistic  behavior 
of  the  3-Satisfiability  Problem  and  found  that,  independent  of  the  size 
of  the  problem,  there  is  only  a  limited  region  of  uncertainty,  i.e.,  only 
a  finite  set  of  values  where  even  the  simplest  sensible  algorithm  can 
make  an  error.  This  problem,  which  is  described  in  detail  below,  is 
representative  of  a  very  large  class  of  difficult  combinatorial  problems 
and  can,  in  fact,  be  used  as  a  vehicle  for  solving  many  other  problems. 
Thus,  we  believe  these  results  can  be  extended  to  other  problems  as 
well,  most  notably,  to  problems  in  network  design  such  as  facility 
location  and  link  topology  optimization.  We  are  currently  in  the  process 
of  investigating  such  extensions  and  of  examining  the  probabilistic 
performance  of  several  specific  heuristic  algorithms. 

C.l.  Probabilistic  Behavior  of  the  3-Satisfiability  Problem 

The  3 -Satisfiability  Problem  has  a  simple  and  homogeneous  struc¬ 
ture  which  makes  it  easy  to  develop  and  study  a  probabilistic  model.  It 
also  does  not  involve  any  numerical  data  which  would  otherwise  obscure 
the  nature  and  generality  of  the  results  which  we  obtain.  These  re¬ 
sults  can,  however,  be  directly  extended  to  optimization  problems  which 
do  involve  numerical  data,  as  we  will  see  in  the  following  discussion. 

In  the  ordinary  satisfiability  problem,  we  are  given  a  Boolean 
expression,  E,  over  m  Boolean  variables  v1,v2,...,vm  and  we  ask  if 
there  exists  a  set  of  truth  values  for  the  variables  which  will  result  in 
E  taking  the  value  True.  Thus,  each  of  the  v.  can  take  either  the 
value  True  or  False  (alternatively  denoted  by  1  or  0)  and  E  will  then 
take  either  the  value  True  or  False  based  on  the  values  of  the  v. .  E  is 
usually  given  in  disjunctive  normal  form,  i.e.,  as  a  set  of  clauses  all  of 
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which  must  be  True  in  order  for  E  to  be  true.  Each  clause  contains 
one  or  more  variables  and  is  said  to  be  true  if  at  least  one  of  the 
variables  in  the  clause  is  assigned  the  value  it  takes  in  the  clause. 
Thus  for  example,  E  =  +  v2)( Vj  +  v3)  has  2  clauses.  The  first, 

(vx  +  v2)  is  true  if  either  vx  is  True  or  v2  is  True.  The  second  is 
True  if  either  vx  is  False  or  v3  is  True.  Thus  E  is  satisfiable  as  the 
values  vt  =  True,  v2  =  True,  and  v3  =  True  satisfy  both  clauses  and 
hence  E.  There  are  several  other  assignments  of  truth  values  vlf  v2, 
and  v3  which  will  satisfy  this  E.  The  expression  (v1)(v1  +  v2)(v2)  on 
the  other  hand  is  not  satisfiable. 

The  3-Satisfiability  Problem  is  a  version  of  the  ordinary  Satisfia¬ 
bility  Problem  where  all  clauses  contain  exactly  3  variables.  Garey  and 
Johnson,  in  their  book  Computers  and  Intractability,  show  this  problem 
is  NP-complete  by  showing  that  any  ordinary  Satisfiability  Problem  can 
be  transformed  into  a  corresponding  3- Satisfiability  Problem  of  roughly 
the  same  size.  Thus  the  two  problems  are  equivalent. 

Many  other  problems  can  also  be  transformed  into  corresponding 
3 -Satisfiability  Problems.  In  particular,  combinatorial  optimization 
problems  such  as  the  Traveling  Salesman  Problem  or  the  problem  of 
locating  earth  stations  in  a  satellite  network  can  be  so  transformed. 
Thus,  we  can  produce  a  Boolean  expression  which  corresponds  to  a 
particular  optimization  problem  in  the  sense  that  if  the  expression  is 
satisfiable  then  the  optimization  problem  has  a  solution  with  a  value  less 
than  or  equal  to  a  given  constant  (for  minimization  problems)  or  greater 
than  or  equal  to  a  given  value  (for  maximization  problems).  Further¬ 
more,  the  satisfying  truth  assignment  can  be  used  to  obtain  the  solution 
to  the  corresponding  optimization  problem  directly.  By  altering  the 
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value  of  the  constant  in  the  above  transformation  the  optimal  solution  to 
the  corresponding  optimization  problem  can  be  found  via  binary  search; 
i.e.,  if  we  know  the  optimum  lies  between  values  Cj  and  c2,  we  try  the 
value  ct  +  c2/2.  This  technique  is  known  as  thresholding. 

One  can  determine  if  an  expression  is  satisfiable  by  assigning  all 
2m  possible  truth  values  to  its  variables.  This  approach  is,  of  course, 
not  practical  for  large  values  of  m.  There  are  many  sophisticated  tech¬ 
niques  for  answering  the  question  whether  or  not  a  given  expression  is 
satisfiable,  but  all  have  running  times  which  ultimately  grow  exponen¬ 
tially  with  the  number  of  variables  and  thus  are  limited  to  problems  of 
modest  size. 

We  obtain  here  a  technique  which  yields  the  a  priori  probability  of 
an  expression  with  a  given  structure  being  satisfiable  given  a  probabil¬ 
istic  model  of  the  space  from  which  the  problem  is  drawn.  For  prob¬ 
lems  where  this  probability,  Ps,  is  very  close  to  1  or  very  close  to 
zero,  we  need  seek  no  further.  Only  for  problems  where  Pg  is  signifi¬ 
cantly  different  from  0  or  1  need  the  question  be  investigated  any 
further.  In  some  cases,  the  Pg  itself  is  all  that  we  need.  For  exam¬ 
ple,  if  we  have  found,  using  a  heuristic  algorithm,  a  solution  of  value 
C[  to  a  given  maximization  problem,  and  have  determined  that  Pg  is  less 
than  .01  for  an  expression  corresponding  to  the  existence  of  a  solution 
of  value  greater  than  or  equal  to  Ci  +  a  (for  a  much  smaller  than  c^, 
then  we  are  reasonably  certain  that  there  is  not  much  to  be  gained  from 
an  attempt  at  further  optimization.  This  is  very  important  because  it 
has  often  turned  out  that  it  is  much  harder  to  verify  the  optimality  (or 
near-optimality)  of  a  solution  than  to  find  an  optimal  solution  especially 
when  the  solution  space  is  rich  and  there  exist  many  alternate  near- 
optimal  solutions. 


We  consider  a  uniform  probabilistic  model  for  the  3-Satisfiability 
Problem  where  all  clauses  are  equally  likely.  Thus  a  problem  is  totally 
characterized  by  v  and  b,  the  number  of  variables  and  clauses,  respec¬ 
tively.  We  will  consider  two  models  which  are  nearly  equivalent.  In 
one  case,  clauses  are  picked  without  replacement,  i.e.,  a  given  clause 
can  appear  at  most  once  in  an  expression.  In  the  other  case,  clauses 
are  picked  with  replacement. 

Duplicate  clauses  do  not  affect  the  satisfiability  of  an  expression. 
We  can  thus  relate  problems  where  the  clauses  are  chosen  with  replace¬ 
ment  to  problems  where  the  clauses  are  chosen  without  replacement  by 
saying  that  a  problem  with  v  variables  and  b  clauses  chosen  with  re¬ 
placement  is  the  same  as  a  problem  with  v  variables  and  b '  clauses 
chosen  without  replacement  if  the  number  of  distinct  clauses  in  the  first 
problem  equals  b'.  The  relationship  between  the  number  of  clauses 
chosen  with  replacement,  b,  and  the  average  number  of  distinct  clauses 
is  given  by  the  following  argument. 

Let  be  the  number  of  distinct  clauses  when  b  clauses  are 
selected  with  replacement.  Then  <  b  and  X^  =  +  a,  where 

a  =  1  with  probability  ,  and  a  =  0  with  probability  1  -  P  .  P 

-  new  ^  1  new  new 

is  the  probability  that  the  b1^  clause  is  different  from  all  the  first 
b  -  1.  P„, ,  is  simply  the  ratio  of  the  number  of  unchosen  clauses  to 

new 

the  total  number  of  possible  clauses.  The  number  of  possible  clauses  is 
B  =  8(3) 

since  each  clause  contains  3  variables  each  of  which  can  take  either  of 

two  values.  Thus  P  is  given  by: 

new  a  1 


P 


new 


B 


B 
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Let  f(b)  be  the  average  number  of  distinct  clauses.  Then 

E<XblXb-l>  =  Xb-1  +  p„ew  (C'2) 

and 

f(b)  =  E(Xb)  =  f(b  -  1)  +  .  (C-3) 

Thus 

f(b)  =  f(b-l)  x^l+i  .  (C-4) 

As  an  example,  if  v  =  10,  then  B  =  960  and  f (50)  =  48.7.  As  we  will 
see,  we  are  most  interested  in  values  of  b  near  5v.  For  v  =  20  and 
b  =  100,  f  =  99.5.  As  v  increases,  f/b  approaches  1  very  quickly. 
Thus,  there  is  little  difference  between  choosing  clauses  with  or  without 
replacement. 

C.2.  Analytical  Method  1 

We  now  turn  to  the  problem  of  estimating  Ps(v,  b),  the  probability 
of  expression  E,  with  v  variables  and  b  clauses,  being  satisfiable. 
Given  a  problem  with  v  variables  and  b  clauses,  we  define  the  truth 
assignment  graph,  G,  associate*  with  that  problem  to  be  a  graph  with 
2V  +  1  nodes.  Nodes  1  through  2V  correspond  to  the  possible  truth 
values  for  the  v  variables  and  node  0  is  a  distinguished  pseudonode. 
Each  clause  in  E  is  inconsistent  with  one  eighth  of  the  truth  assign¬ 
ments,  specifically  those  in  which  ail  three  variables  in  the  clause  have 
values  opposite  to  their  values  in  the  clause.  G  contains  an  arc  be¬ 
tween  node  0  and  any  node  corresponding  to  a  truth  assignment  which 
does  not  satisfy  E.  We  thus  proceed  through  E  clause  by  clause  and 
add  arcs  between  node  0  and  nodes  which  do  not  satisfy  the  clause. 
There  are  2V  ^  arcs  corresponding  to  each  clause.  In  general,  arcs 
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''belonging"  to  different  clauses  overlap,  i.e.,  there  will  be  truth 
values  inconsistent  with  more  than  one  clause.  We  do  not  show  the 
multiplicity  of  these  arcs.  Thus,  G  will  in  general  contain  fewer  than 
b  x  2v”3  arcs.  We  are  interested  in  G  because  P  the  probability  E  is 
satisfiable,  is  precisely  equal  to  the  probability  that  G  is  not  con¬ 
nected.  Figure  C.l  shows  a  truth  assignment  graph. 

The  problem  of  whether  a  random  graph  is  connected  or  not  has 
been  studied  extensively  by  Erdos  and  Renii.  We  will  proceed  along 
similar  lines. 


E  =  (vx  +  v2  +  vG)  (vt  +  v2  t  v4)  (v2  +  v3  +  v4) 
FIGURE  C.l.  A  TRUTH  ASSIGNMENT  GRAPH 


We  define  A(v,  k,  b)  to  be  the  number  of  Boolean  expressions 
whose  corresponding  truth  assignment  graphs  contain  k  or  more  isolated 
nodes  (i.e.,  nodes  with  no  incident  links).  P  (v,  b)  is  then  given  by 

2V 

P  (v,  b)  =  1  -  I  (-l)k  ^  b)  (C-5 ) 

k=0  1  b; 
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where  B  =  8(Y). 

This  relation  follows  from  the  fact  that  if  an  expression  is  satis- 
fiable  then  its  corresponding  truth  assignment  graph  must  contain  at 
least  one  isolated  node.  Note  that  the  alternating  sum  is  required  in 
order  to  account  for  the  situations  where  graphs  with  more  than  k 
isolated  points  are  included  in  terms  with  k  or  more  isolated  points.  We 
seek  the  number  of  expressions  with  graphs  with  one  or  more  isolated 
points.  The  alternating  sum  gives  us  the  number  with  exactly  zero 
isolated  points. 

We  can  write  A(v,  k,  b)  as 

v 

A(v,  k,  b)  =  I  (Y)  21  NM(v ,  i,  k)  B(v,  i,  k,  b)  (06) 

i=0  1 

where  NM(v,  i,  k)  is  the  number  of  ways  of  selecting  k  clauses  which 
match  in  i  specific  variables  (i.e.,  k  clauses  in  which  i  specific  vari¬ 
ables  take  the  same  truth  value  in  all  k  clauses)  and  B(v,  i,  k,  b)  is 
the  number  of  Boolean  expressions  whose  corresponding  truth  assign¬ 
ment  graphs  have  k  or  more  isolated  points  and  the  isolated  points 
match  in  i  variables. 


NM(v,  i,  k)  =  Z  (-1)  (' 

m=0  k 


v-i  „v-i-m  ,  , 

m  ,2  ,  /V-i •>  0m  „  1_  ,0k  0Nv-i 

)  '  )  ^  |  v4*  <*  > 


m 


(07) 


where  the  approximation  is  made  by  replacing 


~v-i-m  /0v-i-mNk 
.z  \  i-..  U- _ ) 


)  by 


Mow,  B(v,  i,  k,  b)  =  (^vh,k)  ^  .v|-iere  C(V,  j(  ;s  the  total 
number  of  clauses  which  are  permissible  in  the  sense  that  they  give 
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rise  to  graphs  that  have  the  requisite  number  of  isolated  points  and 
matched  variables.  c(v,  i,  k)  can  be  estimated  by 


c(v,  i,  k)  s  8(3)  exp[- 


2k  k(v-i)  , 
—  ] 
(2  -2)8v 


(C-8) 


The  details  of  this  approximation,  which  are  somewhat  lengthy,  are 
given  in  W.  Chuang's  thesis. 

Using  the  above  approximations  and  Sterling's  approximation 
n!  s  NN+1*  e'N 

we  find  by  algebraic  manipulation  that 

Ps(v,  b)  =  1  -  exp  [-exp  (-c)]  (C-9) 

where  b  =  8(£n  2V  +  c) 
and  c  is  a  constant. 

Thus,  only  for  b  s  5.5  v  does  P  (v,  b)  take  values  significantly 

different  from  0  or  1  since  for  any  value  of  c  less  than  -2,  Pg  is  nearly 

1  and  for  any  value  of  c  greater  than  +4,  P  is  nearly  0.  Figure  C.2 

s 

summarizes  the  relationship  between  Pg  and  c.  Note  that  this  is  not  a 
function  of  v  or  b  directly.  The  approximations  used  in  obtaining  this 
analytic  form  are,  however,  functions  of  v  as  we  will  see.  The  approx¬ 
imation  turns  out  to  be  reasonable  for  modest  values  of  v  (e.g.,  v  in 
the  range  10  to  30)  which  we  are  interested  in.  Note  especially  the 
sharp  drop  from  1  to  0  over  the  interval  c  =  -2  to  c  =  +4. 


C.3.  Analytic  Method  2 

Because  of  the  approximations  made  in  the  previous  analysis,  we 
sought  a  second  method  to  independently  verify  the  approach.  We 


consider  the  case  now  where  clauses  are  selected  with  replacement.  We 
note  that  all  clauses  are  equally  likely  and  that  a  given  clause  elimi¬ 
nates  1/8  of  the  possible  truth  assignments.  Indeed,  because  of  this 
symmetry,  on  the  average  the  bLn  clause  in  an  expression  eliminates  1/8 
of  the  remaining  truth  assignments  which  satisfy  the  first  b-1  clauses. 
We  say  a  truth  assignment  is  eliminated  if  it  does  not  satisfy  an  expres¬ 
sion.  Thus,  if  we  define  N  (v,  b)  as  the  number  of  truth  assignments 
satisfying  an  expression  with  v  variables  and  b  clauses,  we  have  the 
recurrence  relation 


v-3  2  •  Vv'  b_1)  v-3 

Ng(v,  b)  =  Ns(v,  b-1)  -  2V  d  ^ -  2V  J 


=  |  Ns(v,  b  -  1)  =  (l  )b  N  (v,  0) 


(C-10) 


where  N  is  the  expected  value  of  N  . 

^  b 

Since  N  (v,  0)  =  2V,  i.e.,  all  truth  assignments  satisfy  an  expres¬ 
sion  with  no  clauses,  we  have 


Ns(v,  b)  =  (|)b  2V 


(C-ll) 


P  (v,  b)  is  by  definition  the  probability  that  N  (v,  b)  is  greater 

than  zero.  Unfortunately,  we  only  have  an  expression  for  N  ,  the 

average  value  of  N  and  not  for  the  value  of  N  itself. 

s  s 

If  N  (v,  b)  >>  1,  then  we  would  expect  P  (v,  b)  =  1.  If  N  (v,  b)  <  1, 
^  s  s 

then  we  may  assume  that  NsO,  b)  is  either  0  or  1.  T,,ien 

P-(v,  b)  =  P(N  (v,  b)  =  1)  =  N  (v,  b).  Thus  we  use  the  approximation 


Pg(V,  b)  = 


1  ,  N  (v,  b)  >  1 


N„(v,  b),  N.,(v,  b)  <  1  . 


(C-12) 


This  approximation  is  reasonably  good.  We  first  find  b  such  that 


Ns(v,  b)  =  1: 

(|)b  2V  -  1 

b  =  log28  -V  log;.!  3  5'2  v 
We  then  have 

Ps(V'  b)  =  1  (7}b-5.2v 


b  <  5.2v 
b  >  5.2v 


(C-13) 


(C-14) 


This  function  is  very  similar  to  the  one  obtained  by  the  first 
analytic  method  and  again  the  region  over  which  Ps(v,  b)  drops  effec¬ 
tively  from  1  to  0  is  limited  again  to  a  small  range  and  is  independent 
of  v  and  b.  Figure  C.3  illustrates  the  relationship  between  Pg  and  c 
(where  c  =  b-  q  ' 2 v  ). 

The  dotted  line  of  Fig.  C.3  shows  P  as  found  by  Method  1  for 

s 

the  same  values  of  c.  As  can  be  seen,  for  c  >  1  the  curves  are  vir¬ 
tually  identical.  There  is,  however,  a  5%  difference  between  the  con¬ 
stants  relating  v  and  b  in  the  two  approximations.  Nevertheless,  the 
two  analyses  corroborate  eath  other  and  give  rise  to  the  same  analytical 
behavior  of  P  ;  i.e.,  v  and  b  are  linearly  related  at  the  point  where  P 
drops  and  Pg  decays  exponentially. 


C.4.  Simulation 

Finally,  in  order  to  compare  the  accuracy  of  the  two  analytic 
methods,  we  performed  a  simulation  to  measure  P  (v,  b)  directly.  In 
the  simulation,  clauses  were  generated  at  random  with  replacement,  all 
clauses  being  equally  likely.  Each  time  a  clause  was  generated,  the 
truth  values  it  eliminated  were  eliminated.  This  procedure  was  con- 


tinued  until  no  truth  values  remained.  For  each  value  of  v,  the  value 
of  b  at  which  the  last  remaining  truth  value  was  eliminated  was  re¬ 
corded  and  became  a  sample,  b. .  An  estimate  of  P  (v,  b)  is  then 
formed  by 


(#  of  b.  >  b) 
Ps(v'  k)  =  (total  #  of  b.) 


(C-15) 


Figure  C.4  gives  the  results  of  this  simulation  for  v  =  10,  20  and  30. 
As  can  be  seen,  the  true  value  of  P  (v,  b)  lies  between  the  N  values 
predicted  by  the  two  analytic  methods  and  is  somewhat  closer  to  that 
predicted  by  the  second  method.  The  important  property  of  rapid 
decay  from  1  to  0  is  borne  out. 
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riGURE  C.4  COMPARISON  OF  SIMULATION  AND  TWO  ANALYTICAL  METHODS 


PERSONNEL 


R.  Boorstyn,  Professor 
A.  Kershenbaum,  Associate  Professor 

Students 

Paul  Chu  (Adaptive  Routing) 

Veli  Sahin  (Multihop  Packet  Radio) 

David  Tsao  (New  Switching  Techniques) 
William  Chuang  (Algorithms) 


ACTIVITIES 


The  following  four  Ph.D.  theses  were  successfully  defended. 

Paul  Chu,  A  Dynamic  Routing  Scheme  in  Packet  Switched  Networks 

Veli  Sahin,  Analysis  of  Multihop  Packet  Radio  Networks 

David  Tsao,  Comparison  of  Switching  Techniques 

William  Chuang,  Probabilistic  Analysis  of  Algorithms. 

Our  paper  on  A  Technique  for  Adaptive  Routing  in  Networks 
appeared  in  the  April  1981  issue  of  IEEE  Transactions  on  Communica¬ 
tions,  Special  Issue  on  Congestion  Control  in  Computer  Networks. 

A  paper  on  Generalized  Augmenting  Paths  was  presented  at  the 
10th  IFIPS  Conference  on  System  Modeling  and  Optimization  in  New 
York,  August  1981.  The  paper  will  appear  in  the  proceedings  to  be 
published  later. 

A  paper  on  a  Simulation  of  a  Dynamic  Routing  Scheme  will  be 
presented  at  the  IEEE  NTC  '81  Conference  in  New  Orleans,  November 
1981. 

A  talk  on  Dynamic  Routing  was  given  at  Carleton  University, 
Toronto,  Canada  at  an  all-day  seminar  on  Networks  co-sponsored  by 
Carleton  University,  IEEE  Toronto  Chapter,  and  Bell  Northern  Re¬ 
search. 
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APPENDIX 


A  Shortest  Path  Algorithm  for  the  Solution  of  the  Simple  Knapsack 
Problem  and  Extensions. 


A  paper  to  be  submitted  for  publication. 
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A  SHORTEST  PATH  ALGORITHM  FOR  THE  SOLUTION 
OF  THE  SIMPLE  KNAPSACK  PROBLEM  AND  EXTENSIONS 


Aaron  Kershenbaum* 


Department  of  Electrical  Engineering  and  Computer  Science 
Polytechnic  Institute  of  New  York 
Brooklyn,  NY  11201 


ABSTRACT 

We  consider  several  versions  of  the  Knapsack  Problem  and  show 
that  they  can  be  solved  using  a  shortest  path  algorithm  requiring  both 
storage  and  running  time  which  are  polynomial  functions  only  of  the 
number  of  types  of  object  and  the  size  (weight)  of  a  single  object. 
This  is  a  significant  improvement  over  previous  algorithms  for  the 
solution  of  the  Knapsack  Problem  using  shortest  paths,  which  typically 
require  storage  and  runtime  which  are  functions  of  the  total  knapsack 
size. 

I.  INTRODUCTION 

The  Knapsack  Problem,  which  is  interesting  in  its  own  right,  has 

131 

also  received  much  attention  recently1  1  as  a  means  of  solving  integer 
programming  problems  via  a  relaxation  technique  using  group  theory. 
Formally,  the  Knapsack  Problem  can  be  stated  as: 

M 

Maximize  z  =  I  p.x. 

i=l  1  1 

*This  work  was  partially  supported  by  NSF  under  Grant  ENG  7908120 
and  by  U.S.  Army  CECOM  under  Contract  DAAK-80-80-K-0579 . 
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M 

subject  to  =  1  wx.  =  W 
i=l  1  1 

Xj  >  0  for  i  =  1 , 2 , . . .  M 
x.  integer 

Thus,  we  are  given  M  types  of  object  with  weight  w.  and  profitability 
p.  per  object  of  type  ..  We  are  required  to  maximize  the  total  profit 
subject  to  a  constraint  that  the  sum  of  the  weights  of  the  objects 
selected  equals  W.  In  the  simple  (unbounded)  version  of  the  problem 
there  is  no  restriction  on  the  number  of  objects  of  each  type  which 
may  be  included  in  the  solution. 

Sometimes  an  inequality  constraint  is  used  in  place  of  the  equal¬ 
ity  constraint,  i.e.,  one  seeks  objects  whose  aggregate  weight  does 
not  exceed  W.  Such  a  problem  can  be  transformed  into  a  problem 
with  an  inequality  constraint  by  including  an  additional  object  with 
unit  weight  and  zero  profitability.  Here  we  will  concentrate  primarily 
on  problems  with  equality  constraints. 

We  assume  that  W  and  the  w.  are  non-negative  integers  (or, 
more  generally,  that  they  are  commensurate)  and  that  the  p^  are 
non-negative. 

Previous  approaches  to  the  solution  of  this  problem  include  those 
reported  by  Horowitz  and  Sahni^  using  dynamic  programming  and 
implicit  enumeration.  These  approaches  have  been  found  to  be  effec¬ 
tive  for  a  wide  range  of  problems  but  have  exhibited  excessive  run¬ 
times  for  problems  with  a  large  number  of  neariy  equally  valuable  (in 
terms  of  cost  per  unit  weight)  objects.  Also,  their  worst  case  run¬ 
ning  times  are  exponential  in  the  number  of  object  types.  Denardo 
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f2 1 

and  Fox1  -  have  developed  an  approach  which  overcomes  these  ob¬ 
jections  by  solving  the  problem  as  a  shortest  path  problem  in  a  net¬ 
work  with  W  nodes  and  W  x  M  arcs.  Thus,  their  approach  requires 
storage  proportional  to  W  and  runtime  proportional  to  W  x  m.  If  W 
and  M  are  sufficiently  small,  their  approach  is  extremely  attractive. 
If  W  is  large,  however,  the  storage,  and  to  a  lesser  extent  the  run¬ 
time,  become  prohibitive. 

Denardo  and  Fox  present  their  algorithm  in  the  context  of  using 
a  Knapsack  Problem  as  an  integer  programming  relaxation.  In  such 
cases,  W  is  generally  of  the  same  order  as  the  w. .  We  extend  their 
approach  to  the  case  where  W  is  very  large  but  the  individual  w^  are 
not,  and  exploit  the  underlying  cyclic  group  structure  of  the  problem 
to  develop  an  algorithm  with  both  runtime  and  storage  requirements  a 
function  only  of  the  weight  of  a  single  object  and  the  number  of 
object  types.  Garfinkle  and  Nemhauser^  point  out  the  relationship 
to  the  group  structure  but  do  not  discuss  application  of  the  structure 
to  develop  an  algorithm  as  efficient  as  the  one  presented  here. 


II.  BASIC  PROCEDURE 

For  the  sake  of  clarity,  we  begin  by  describing  the  basic  proce¬ 
dure  and  justifying  it.  In  later  sections,  extensions  and  refinements 
of  this  procedure  are  considered. 

We  begin  by  defining  a  network  K  =  ( N ,  A ,  L )  corresponding  to 
the  Knapsack  Problem  (KP).  The  node  set,  N,  contains  W  +  l  nodes 
numbered  0  through  W  corresponding  to  the  aggregate  weight  of  the 
objects  selected  thus  far.  At  each  node,  i,  there  are  outgoing  arcs 
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corresponding  to  feasible  objects  which  may  be  selected  to  augment  a 
partial  solution  with  weight  i.  Thus,  at  node  i  there  will  be  arcs 

(i,  i  +  w^)  for  all  K  such  that  i  +  w^  <  W.  The  length  of  an  arc 

corresponding  to  an  object  of  type  K  is  p^.  We  thus  have: 

N  =  (0,1,...  W} 

A  =  {(i,  i  +  WjP?  i  =  0,1,... W,  for  all  k  such  that  i  +  <  Wj 

L  +  (hj  '  (i,j)eA,  where  1..  =  p,  for  arcs  corresponding  to  type 

1J  lj  K 

k  objects} 

The  network  thus  contains  approximately  MW  arcs.  We  will  assume 

that  w,  r  w,  for  kx  f  k2  as  if  two  types  of  objects  have  the  same 
K1  K  2 

weight  the  one  with  the  smaller  profit  can  be  ignored.  We  also  assume 
w^,  >  0  and  p^  >  0  for  all  k.  We  will  refer  to  arcs  corresponding  to 
a  type  k  object  simply  as  type  k  arcs.  Figure  1  illustrates  part  of  a 
network  corresponding  to  a  KP  with  4  types  of  object,  described  in 
the  table  accompanying  Figure  1.  For  the  sake  of  clarity,  only  nodes 
0  through  7  and  arcs  from  nodes  0  and  1  are  shown  in  Figure  1. 

Paths  from  node  0  to  node  W  in  such  networks  correspond  to 
solutions  of  the  associated  KP.  The  length  of  a  path,  defined  as  the 
sum  of  the  lengths  of  the  arcs  in  the  path,  is  the  total  profit  of  the 
solution.  Thus,  the  longest  path  corresponds  to  the  optimal  solu¬ 
tions.  Note  that,  by  the  assumption  that  w^  >  0  for  all  k,  the  net¬ 
work  is  acyclic  and  so  the  problem  of  finding  the  longest  path  from  0 
to  W  is  solvable  using  Bellman's  shortest  path  algorithm  suitably  modi¬ 
fied  to  find  longest  paths.  Defining  d^  to  be  the  length  of  the  cur¬ 
rent  estimate  of  the  longest  path  from  node  0  to  node  i,  this  amounts 
simply  to: 

that  w,  t  w,_  for  K,  ?  Ko  as 


,  Algorithm  1 

Step  0:  d0  <-  0 

d .  ■*-  oo  i  =  1 , . . . .  W 

I  Step  1:  For  i  =  1  to  W 

For  all  j  such  that  (i,j)eA 
(dj  =  max  (dj,dj  +  l.j) 

|  One  can  also  keep  track  of  the  solution  itself  by  setting  a^  «-  i  when¬ 

ever  the  value  of  d.  is  updated.  At  the  end  of  the  algorithm  one  can 
trace  the  path  back  from  node  W  to  node  0  using  the  a^.  We  refer  to 
I  dj  as  the  label  on  node  j  and  to  aj  as  the  predecessor  of  node  j. 

Step  1  is  known  as  a  scan  of  each  node,  i. 

Algorithm  1  is  the  simplest  possible  shortest  path  algorithm. 

|  The  nodes  are  scanned  in  numerical  order  and  each  node  is  scanned 

only  once.  The  algorithm  has  a  running  time  on  the  order  of  MW 
operations  and  a  storage  requirement  on  the  order  of  W.  If  W  is 
|  very  large,  the  approach  becomes  unattractive.  The  approach  des¬ 

cribed  below  avoids  this  problem  by  solving  the  problem  using  a 
network  which  is,  in  general,  much  smaller.  The  key  to  reducing  the 
I  size  of  the  network  is  embodied  in  the  following  lemmas.  Define 


I 


I 


*  H 

w  =  max  (w^)  and  w*  =  w.  such  that  =  max  _k.  Thus  w  is  the 
k  J  k  Wk 

weight  of  the  "heaviest"  type  of  object  and  w*  is  the  weight  of  the 

"best"  type  of  object. 

The  proofs  of  some  of  the  lemmas  rely  upon  elementary  proper¬ 
ties  of  cyclic  groups.  The  interested  reader  is  referred  to  [6]  for 
more  details. 
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Lemma  1:  An  optimal  KP  solution  exists  containing  no  set  of 
objects  whose  aggregate  weight  is  a  multiple  of  w*. 

Proof :  Given  an  optimal  KP  solution  containing  a  set  of  objects 
of  aggregate  weight  nw*,  we  can  replace  these  objects  by  n  "best" 
objects  without  altering  the  total  weight.  The  net  change  is  profit, 
Ap  is  defined  by: 


Ap  =  np*  -  I  p. 

j  J 


By  the  definition  of  p*,  each  term  of  the  sum  is  non-negative 
and  hence  Ap  >  0.  We  can  thus  replace  the  set  without  destroying 
optimality  and  the  lemma  is  proven. 

The  terms  in  the  above  sum  can  be  interpreted  as  the  amount  of 

profit  "wasted"  by  using  a  type  j  object  instead  of  a  best  object.  We 

can  in  fact  redefine  the  arc  lengths  in  the  network  corresponding  to 

a  given  KP  to  be  precisely  these  quantities.  The  optimal  KP  solution 

then  corresponds  to  the  shortest  path  from  0  to  W.  This  can  be  seen 

to  be  true  by  observing  that  the  above  transformation  of  the  arc 

lengths  affects  the  lengths  of  all  paths  from  0  to  W  in  precisely  the 

same  way.  If  the  length  of  a  0  to  W  path  in  the  original  network  was 

W 

p  the  length  of  the  same  path  in  the  new  network  is  p*  —  *  -  p 
Thus  the  relative  length  of  all  0  to  W  paths  are  reversed.  Alterna¬ 
tively,  one  can  observe  that  the  optimal  path  is  the  one  which  wastes 
the  least  profit.  In  the  sequel,  we  use  this  form  of  the  length  func¬ 
tion  as  it  will  allow  us  to  improve  the  efficiency  of  the  algorithm.  We 
refer  to  the  network  using  this  length  function  as  Kx. 
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In  Figure  1,  the  new  lengths  of  arcs  of  types  1  through  4  are 
1,2,0,  and  2,  respectively.  Note  that  type  3  arcs  are  best  and  that 
best  arcs  have  0  length.  All  arcs  have  non-negative  length.  If  the 
best  type  of  arc  is  not  unique  (e.g.  if  w2  =  9  in  the  above  example), 
it  is  possible  for  the  length  of  more  than  one  type  of  arc  to  be  0. 
To  avoid  confusion,  we  assume  that  the  best  type  of  object  is  unique. 
The  extension  of  the  algorithms  to  this  case  is  straight  forward. 

Lemma  2:  An  optimal  solution  exists  containing  no  more  than 
w*  -  1  non-best  objects. 

Proof :  Any  set  of  w*  or  more  objects  must  contain  a  subset  of 
weight  nw*  for  some  n  >  0.  To  see  this,  consider  a  set  of  w*  ob¬ 
jects,  i1,i2, . .  -iw*-  From  the  sums 
i 

S.  =  (  2  W.  )  modulo  W* 

J  k=l  1  k 

i 

If  S-  =  0  then  2  W.  =  nw*. 

J  k=l  1  k 

£ 

If  S.  =  S0  for  £  >  j  then  2  W.  =  nw*. 

]  *  k=j  1  k 

But  there  are  w*  S.'s  and  only  w*  -  1  nonzero  values  of  modulo 

w*.  Hence,  either  S.  =  0  or  S.  =  S0  for  some  £  and  the  set  contains 

a  subset  of  weight  nw*.  From  the  proof  of  Lemma  1  we  see  that  we 

can  replace  any  such  subset  by  n  best  elements  without  decreasing 
the  profit.  Thus,  we  need  only  consider  sets  containing  w*  -  1  or 
fewer  non-best  elements  together  with  zero  or  more  best  elements. 
An  optimal  solution  can  always  be  obtained  L  this  way. 

But  since  ail  elements  have  weight  no  greater  than  w,  the  ag¬ 
gregate  weight  of  the  non-best  elements  in  any  such  optimal  solution 


will  not  exceed  w*(w*  -  1).  If  W  >  ww* ,  we  need  not  consider  W 
explicitly  as  a  constraint.  We  need  simply  apply  the  above  algorithm 
to  get  dj  for  j  <  ww*,  and  then  complete  the  solution  by  adding  the 
appropriate  number  of  best  objects. 

Indeed,  we  are  only  interested  in  the  weight  modulo  w*  of  a 
partial  solution  since  any  two  solutions  whose  weights  differ  by  a 
multiple  of  w*  differ  only  by  one  or  more  best  objects.  We  thus 
have: 

Lemma  3:  If  W  >  ww*,  the  optimal  KP  solution  can  be  found  by 
adding  zero  or  more  best  objects  to  the  set  of  objects  defining  the 
shortest  path  in  the  network  formed  by  merging  all  nodes  with  the 
same  weight  modulo  w*  as  described  above. 

Proof :  Consider  the  original  network  before  the  nodes  were 

merged.  Consider  the  0  to  W  path  corresponding  to  an  optimal  solu¬ 
tion.  By  Lemma  1,  we  may  assume  this  path  contains  no  set  of  arcs 
(correspond  to  objects)  of  weight  nw*  except  for  best  arcs.  Since 
the  network  contains  paths  corresponding  to  picking  these  objects  in 
all  possible  orders,  we  may  assume  that  the  path  under  consideration 
has  all  the  best  arcs  at  the  end.  Consider  only  the  initial  subpath 
corresponding  to  the  non-best  arcs. 

Now,  consider  what  the  transformation  does  to  this  initial  sub¬ 
path.  All  the  nodes  in  this  path  have  different  weights  modulo  w*. 
Thus  the  transformation  changes  this  optimal  0  to  W  path  in  Kx  to  a  0 
to  W'  path  in  the  network  with  merged  nodes,  where  W'  =  W  modulo 
w*.  Note  that  other  paths,  including  some  optimal  ones,  become 
complex  paths  containing  cycles  because  they  contain  subsets  whose 
aggregate  weights  are  a  multiple  of  w*.  The  path  we  are  focusing  on 
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however  remains  a  simple  path  and  can  thus  be  found  using  an  ordi¬ 
nary  shortest  path  algorithm. 

No  new  paths  are  created  by  the  transformation.  Thus,  an 
optimal  KP  solution  can  be  obtained  by  appending  0  or  more  best  arcs 
to  the  shortest  0  to  W'  path  in  the  new  network,  and  the  lemma  is 
proven. 

It  is  thus  possible  to  define  a  new  network 
K2  =  (N,A,L): 

N  =  {0,1, . .  .w*-l} 

A  =  {(i,j)  |  j  =  (i  +  wk)  modulo  w*  for  some  K} 

wk 

L  =  {£j,  |(i,j)s  A,  a...  =  —  p*  -  pk  for 
u  w*  K 

k  defining  (i , j) } 

This  network  has  only  w*  nodes  instead  of  W.  Furthermore,  we 

note  that  if  (W,  =  W,  )  modulo  w*,  then  type  kx  and  type  k2  arcs 
K1  k2 

are  parallel  and  the  arc  corresponding  to  the  object  with  the  smaller 
Jl.j  can  be  ignored.  Thus,  there  is  at  most  1  outgoing  arc  from  each 
node  to  each  other.  The  total  number  of  arcs  is  thus  at  most 
w*(w*  -  1).  We  have  thus  removed  dependency  on  both  M  and  W  and 
have  a  procedure  whose  running  time  and  storage  are  a  polynomials 
only  in  w*. 
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The  following  is  the  essence  of  the  procedure: 

Step  0:  (Initialization) 
d0  =  ° 

dj  =  »  j  =  1 , . . .  w*-l 

where  dj  is  the  current  estimate  of  the  length  of  the  shortest  path 
from  0  to  j. 

Step  1:  Find  i,  the  best  node  to  scan  next.  This  is  discussed  in 

detail  below. 

Step  2:  (Scan  node  i) 

For  k  =  2 ,  . . .  M 
Let  j  =  (i  +  w.  )  modulo  w* 

If  dj  >  di  +  £. j  then  dj  =  dj  +  9.^  and  PRj  =  k 

where  £jj  is  the  length  of  an  arc  corresponding  to  a  type  k  object 

(JLj  =  wk(p*/w*  -  pk)  as  defined  above)  and  PRj  records  the  type  of 
the  arc  used  to  label  node  j  and  hence  keep  track  of  the  optimal 

solution . 

Step  3:  Return  to  Step  1  if  any  nodes  remain  to  be  scanned;  other¬ 
wise  stop. 

It  is  clear  that  the  storage  required  for  this  procedure  is  linear 
in  w*  +  M.  If  Step  1  is  sufficiently  simple  and  each  node  is  only 

scanned  once,  the  procedure's  runtime  will  be  dominated  by  Step  2 

and  will  be  proportional  to  Mw*.  This  is  in  fact  the  case,  as  we  will 

I  see. 

A  node,  j,  must  be  scanned  if  its  label,  dj,  is  improved  (re¬ 

duced)  as  it  may  then  improve  the  labels  of  other  nodes.  If  we  are 
to  ensure  that  each  node  is  scanned  at  most  once,  we  must  defer 
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scanning  it  until  its  label  cannot  be  improved.  When  all  arc  lengths 
are  positive,  as  they  are  here,  this  objective  is  achieved  by  scanning 
the  nodes  in  ascending  order  of  dj.  This  is  Dijkstra's  well  known 
shortest  path  algorithm. 

In  the  general  case,  Dijkstra's  algorithm  requires  that  in  Step  1 
we  find  the  as  yet  unscanned  node  with  the  smallest  label.  If  this 
were  done  naively,  it  would  require  an  examination  of  w*  labels  and 
the  overall  procedure  would  require  (w*)2  +  Mw*  operations.  In  this 
case,  however,  we  can  implement  Step  1  carefully  and  do  much  bet¬ 
ter. 

We  form  "buckets"  of  nodes  to  be  scanned  and  place  nodes  to  be 
scanned  together  in  the  same  bucket  if  they  have  similar  labels.  If 
the  width  of  the  buckets  is  q,  then  we  place  a  node  with  label  d, 
where  (k-1)  <  d  <  kq  in  the  kth  bucket.  We  then  simply  work  our 
way  through  the  buckets  in  ascending  order  of  bucket  number  scan¬ 
ning  the  nodes  in  each  bucket  in  arbitrary  order.  Thus,  there  is  no 
explicit  search  in  Step  1.  If  the  width  of  the  bucket  is  no  greater 
than  the  length  of  the  shortest  arc  in  the  graph ,  then  any  node 
labeled  by  the  node  currently  being  scanned  will  reside  in  a  higher 
numbered  bucket  and  hence,  each  node  will  be  scanned  at  most  once. 

The  only  problem  with  this  procedure,  which  is  well  known,  is 
that  the  number  of  buckets  itself  may  become  very  large,  in  particu¬ 
lar  much  larger  than  the  number  of  nodes,  if  the  longest  arc  is  many 
times  greater  than  the  shortest.  In  this  case,  a  great  deal  of  storage 
and  running  time  may  be  wasted  in  dealing  with  empty  buckets.  We 
now  show  that  in  this  case  fewer  than  w*  buckets  are  required  in  all 
cases. 
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We  begin  by  reindexing  the  arcs  out  of  each  node  corresponding 
to  the  M  -  1  remaining  types  of  object  in  increasing  order  of  length. 
(Recall  that  the  arc  corresponding  to  the  best  type  of  object  was  re¬ 
moved  from  explicit  consideration.)  Thus,  L1(  is  the  length  of  the 
arc  which  corresponds  to  the  second  best  type  of  object,  etc.  The 
w.  and  p.  are  reindexed  correspondingly.  We  will  refer  to  arcs  of 
length  L-,  weight  w.,  and  profit  p.  after  this  reindexing  as  arcs  of 
type  i  (or  as  arcs  corresponding  to  objects  of  type  i).  For  sim¬ 
plicity,  we  will  assume  L.  t  Lj  for  i  f  j  although  the  procedure  does 
not  require  this. 

Consider  the  case  where  w*  is  relatively  prime  to  wx.  By  the 
definition  above,  arcs  of  length  Lx  are  the  shortest  arcs  in  the 
graph.  Since  wx  and  w*  are  relatively  prime,  a  path  containing  at 
most  w*  -  1  arcs  comprised  entirely  of  arcs  of  type  1  exists  from  0  to 
each  other  node.  Thus,  we  know  that  no  node  need  have  a  label 
greater  than  Lx(w*  -  1).  So,  w*  -  1  buckets  of  width  Lx  suffice 
since  the  shortest  path  will  be  no  longer  than  the  aforementioned  one. 

In  the  case  where  w*  and  wx  are  not  relatively  prime,  the  above 
path  will  loop  back  to  node  0  before  reaching  many  of  the  other 
nodes.  In  this  case,  the  following  preliminary  computation  is  carried 
out  before  setting  up  the  buckets: 

r  =  w* 

For  j  =  1,  M-l 

g.  =  r/GCD(r,Wj) 
r  =  GCD(r,Wj) 

where  GCD(i,j)  is  the  greatest  common  divisor  of  the  integers  i  and  j 
and  can  be  found  using  Euclid's  Algorithm,  whose  running  time  is 
bounded  by  the  following  Lemma: 


Lemma  4:  Euclid's  Algorithm  has  a  worst  case  running  time  of 
order  log  N  when  applied  to  find  the  GCD  of  N  and  M,  for  N  >  M. 

Proof:  For  N  >  M,  Euclid's  Algorithm  replaces  a  problem  on  N 
and  M  by  one  one  M  and  N  -  qM.  The  worst  case  for  this,  i.e.,  the 
one  which  converges  most  slowly,  is  when  q  =  1  at  all  stages.  This 

th 

results  in  a  Fibonacci  sequence.  It  is  well  known  [1]  that  the  K 
Fibonacci  number,  F^  is  given  by: 


FK  V5 


1 
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Thus,  the  runtime  of  the  algorithm  is  of  order  K  for  N  of  order 
Fj,,  i.e.,  K  is  of  order  log  N. 

We  observe  that,  corresponding  to  the  paths  comprised  entirely 
of  type  1  arcs  above,  there  is  now  a  tree  rooted  at  0,  spanning  all 
the  nodes,  and  containing  paths  with  at  most  g.  -  1  arcs  of  length 
L- .  An  illustration  of  such  a  tree  is  given  in  Figure  4.  In  this  case 
only  nodes  0i8,  and  4  can  be  reached  using  only  arcs  of  type  1 

before  the  path  cycles  back  to  node  0.  The  number  of  nodes  so 

reachable  is  gt  (three  in  this  case).  The  original  set  of  w*  nodes  is 
partitioned  into  gj  parts  which  are  then  each  partitioned  into  g2 
smaller  parts  corresponding  to  the  nodes  which  are  now  reachable  via 
arcs  of  type  2.  The  partitioning  continues  until  all  nodes  are 

reached.  The  above  can  be  proven  rigorously  in  the  general  case 

using  the  properties  of  cyclic  groups. 

We  now  define  buckets  of  variable  width  corresponding  to  the 
longest  path  in  the  tree  defined  above.  Thus,  there  are  gi  -  1 
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buckets  of  width  lx  followed  by  g2  -  1  buckets  of  width  12,  etc.  A 
total  of  B  =  ^  (g.  -  1)  buckets  is  required.  Since  the  product  of  the 
g.'s  is  w*  B  <  w*. 

We  have  shown  that  less  than  w*  buckets  are  required.  We  now 
show  that  no  node  is  scanned  more  than  once.  To  do  so,  we  show 
that  the  width  of  each  bucket  is  no  greater  than  the  length  of  the 
shortest  arc  still  permitted  in  a  path. 
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Figure  4:  A  spanning  tree  bounding  the  lengths  of  shortest  paths 


Note  that  the  paths  from  0  to  j  correspond  to  ordered  selections 
of  objects  of  aggregate  weight  j;  i.e.,  there  are  distinct  paths  cor¬ 
responding  to  the  selection  of  the  same  objects  in  different  orders. 
We  can,  and  should,  consider  only  one  path  corresponding  to  each 
distinct  collection  of  objects.  This  is  easily  done  by  keeping  track  of 
the  last  type  of  arc  in  the  path  (which  we  are  doing  with  the  variable 
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PR.)  and  only  considering  arcs  with  the  same  or  higher  index  (object 
type).  Thus,  we  will  find  solutions  corresponding  to  a  selection  of 
objects  in  non-decreasing  order  of  type. 

Now,  consider  a  node  in  bucket  i  of  width  Wj.  This  node  was 
either  reached  via  a  path  in  the  tree  considered  above  or  via  an  arc 
whose  length  is  greater  than  that  of  any  in  that  tree.  In  latter  case, 
the  length  of  the  smallest  arc  under  consideration  exceeds  the  width 
of  any  bucket.  In  the  former  case,  since  we  are  considering  arcs  in 
increasing  order  of  length,  again  a  node  can  only  enter  a  bucket  via 
an  arc  at  least  as  long  as  the  width  of  the  bucket. 

Extensions 

If  W  <  w*w  then  it  is  possible  for  w.  to  be  equal  to  w^  and  for 
objects  i  and  j  not  to  dominate  one  another.  In  particular,  if  w.  >  w^ 
and  lb  <  neither  i  nor  j  may  dominate.  This  is  because  the  con¬ 
straint  on  W  may  prohibit  the  use  of  a  sufficient  number  of  objects 
with  the  smaller  length.  As  an  example  of  this,  consider  a  problem 
with  W  =  24  and  (W^P.)  of  (5,50),  (6,58),  (11,109),  (1,0).  Here, 

the  are  0,2,1,  and  10,  and  w2  =  w3  modulo  5.  Neither  object  2 
nor  object  3  dominates  the  other,  however.  In  fact,  the  optimal 
solution  is  4  type-2  objects  in  this  case.  For  W  =  21  or  W  =  22, 
however,  type-3  objects  would  be  used  in  the  optimal  solution. 

For  W  =  23,  both  type-2  and  type-3  objects  would  be  used.  Lemma 

3  can  thus  not  be  extended  directly  to  the  case  whose  W  <  w*w. 

Thus,  it  is  not  possible  to  make  the  final  reduction  to  a  network 
with  w*  nodes  in  this  case.  However,  when  the  individual  w.  are 
small,  which  is  the  principal  case  of  interest  here,  w*w  will  be  small 
as  well.  The  above  algorithm  can  then  still  be  applied  effectively  to 
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the  network  with  W  nodes.  Alternatively,  one  can  view  the  following 
procedure  as  working  on  the  reduced  graph  with  w*  nodes  but  allow¬ 
ing  multiple  labels  on  each  node.  We  adopt  this  latter  point  of  view 
in  describing  the  modified  procedure  below. 

A  label  on  a  node,  i,  is  now  a  couple  (d.,  c.)  where  d.  is  the 
amount  of  wasted  profit  to  get  to  node  i,  i  =  1,2,...  w*-l,  and  c.  is 
the  cycle  number  of  this  label.  The  c.  are  defined  by  the  relation 
i  =  j  -  w*c. 

where  j  is  the  node  in  the  network  Kx  (with  W  nodes)  which  would 
have  received  the  same  label.  A  node  i  in  K2  can  thus  have  several 
labels  corresponding  to  different  c. ,  but  for  any  two  labels  (d.  ,c.) 
and  (d-',cp  on  the  same  node  i, 

d/  >  d.  -*■  c'  <  c. 

1  1  l  l 

since  as  we  noted  above  the  only  reason  for  considering  a  larger  d. 
would  be  to  obtain  a  smaller  c..  Similarly,  for  any  pair  of  objects 
with  weights  having  the  same  residue  modulo  w*  the  object  with  the 
larger  weight  must  have  the  smaller  length. 

There  are  several  limitations  on  the  maximum  number  of  objects 
of  any  given  type.  In  particular,  m. ,  the  maximum  number  objects  of 
type  i,  is  bounded  by 


m.  <  min 

l  - 


W_ 

w. 


w* 


'  GCD(w*,r.) 


where  r.  is  the  residue  of  w.  modulo  W*.  The  first  term  on  the  right 
i  l  3 

hand  side  follows  from  the  fact  that  any  larger  number  of  type  i 
objects  have  weight  greater  than  W.  The  second  term  follows  by  the 
some  reasoning  is  Lemma  2. 
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Proceeding  along  the  lines  similar  to  those  in  [3]  we  create 

t_ 

(log2  M.)  +  1  objects  corresponding  to  1,2, 4,8,...  and  2°  objects  of 

type  i,  where  b  is  largest  power  of  2  that  is  less  than  M..  We  thus 

now  have  defined  a  new  problem  with  at  most  M  log2  W  objects.  At 
most  one  object  of  each  type  is  permitted  in  the  solution. 

Proceeding  along  lines  similar  to  those  in  [3],  we  can  create 
(log2  M.)  +  1  objects  corresponding  to  1,2, 4, 8... 2^  objects  of  type 

i,  where  b  is  the  largest  power  of  2  that  is  less  than  M..  We  then 

have  a  problem  with  at  most  M1  =  Mlog2(w*)  objects  where  at  most 
one  object  of  each  type  need  be  considered.  Thus,  we  could  create  M1 
buckets  and  proceed  with  the  algorithm  as  above. 

One  can  refine  this  procedure  by  eliminating  dominated  objects 
from  consideration.  In  particular,  if  two  objects  i  and  j,  have  the 
same  residue  modulo  w*,  and  w.  >  w^  and  J£.  <  then  object  j  may  be 
eliminated  from  further  consideration.  One  must  be  careful,  however, 
not  to  allow  one  copy  of  an  object  to  eliminate  another  copy  of  the 
same  object. 

For  example  suppose  there  were  initially  4  types  of  object  with 
(W.,P.)  equal  to  (10,20),  (3,5),  (7,4)  and  (1,0)  W  =  10  and  W  <  ww*, 
respectively.  If  W  =  37  we  must  consider  the  modified  procedure. 
We  thus  create  objects  with  (W.,Pp  equal  to  (6,10),  (12,20),  and 
(6,10)  corresponding  to  2,4,  and  2  objects  of  type  2.  These  three 
new  objects  together  with  the  original  object  of  type  2  can  be  used  to 
select  anywhere  from  0  to  9  copies  of  the  type  2  object.  The  first 
new  object  is  identical  to  the  third  and  would  dominate  it  unless  we 
specifically  prohibited  this. 
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If  the  W.  and  P.  are  drawn  independently  from  uniform  distribu¬ 
tions,  this  latter  refinement  will  reduce  the  average  number  of  objects 
to  no  more  than  w*£n(M)  objects  and  buckets  as  we  see  from  Lemma 
5 . 

Lemma  5:  There  will  be  on  the  average  on  the  order  of  no  more  than: 

W*  £n 

objects  left  undominated  after  dominated  objects  are  removed  in  the 
above  procedure. 

Proof :  As  observed  above,  there  are  approximately  Mlog2(W*)  or 

fewer  objects  before  dominance  is  checked.  Suppose  these  objects 
divide  evenly  into  residue  classes.  There  would  then  be  Mlog2(w*)/w* 
objects  in  each  residue  class. 

An  object  can  be  dominated  by  any  other  object  in  the  same 
class.  If  the  objects  are  ordered  in  increasing  order  of  i 2.,  an  object 
will  be  dominated  unless  its  w.  is  less  than  the  w.  of  all  objects  pre¬ 
ceding  it  on  the  list  since  the  weights  are  uniformly  distributed,  the 
item  in  the  list  will  be  undominated  with  probability  Thus, 
given  a  list  of  length  L  the  expected  number  of  undominated  elements 
remaining  on  the  list  is 
L 

I  1 

j=l  j  £n(L) 

The  result  follows  directly  by  substituting  for  L.  If  the  residue 
classes  do  not  all  contain  the  same  number  of  elements,  the  result  is 
still  true  since  we  observe  that  the  quantity 
iln(Lj)  ♦  £n(L2)  for  L!  +  L2  =  2L 
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is  maximized  for  Lx  =  L2  =  L,  that  is,  equal  sized  residue  classes 
yield  the  maximum  number  of  objects. 

Thus,  even  for  W  <  w*w,  we  have  a  number  of  buckets  polynomial 
in  w*,  not  W.  The  running  time  is  thus  at  worst  of  orderWw*  <  (w*)2w. 

It  is  possible  to  extend  the  procedure  still  further  to  take  into 
account  restrictions  on  the  number  of  permissible  objects  of  each 
type,  for  some  or  all  of  the  objects.  First,  if  one  is  given  an  a 
priori  restriction,  U-,  on  the  number  of  objects  of  type  i,  one  need 
only  set 

M.  =  Min  (M.,U.) 

where  M.  is  the  maximum  number  of  objects  of  type  is  permitted  and 
the  M.  on  the  right  hand  side  is  computed  as  above.  If  the  a  priori 
maximum  permissible  number  of  best  objects  is  less  than  W/w*  than  a 
fictitious  best  object  with  w*  =  W  +  1  must  be  added  to  the  problem. 
This  maintains  the  validity  of  the  procedure  but  may  reduce  its 
efficiency  considerably  if  the  original  w*  was  much  smaller  than  W. 


CONCLUSIONS 

We  have  presented  an  algorithm  for  the  solution  of  the  simple 
(unbounded)  Knapsack  Problem  whose  running  time  and  storage  are 
functions  only  of  the  size  of  the  "best"  object  weight,  W*  in  the  case 
where  the  overall  weight  constraint  is  sufficiently  large.  In  the  cases 
where  the  overall  weight  constraint  is  smaller  or  where  restrictions 
exist  on  the  number  of  objects  of  a  given  type,  we  present  an  exten¬ 
sion  of  this  procedure  which  in  practice  is  often  very  efficient  and 
whose  worst  case  running  time  is  no  greater  than  (w*)2w  where  w  is 
the  weightiest  object. 
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